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00 (54) Title: OBJECT AND METHOD FOR CALIBRATION OF A THREE-DIMENSIONAL LIGHT SCANNER 

\o 

^ (57) Abstract: The invention relates to the field of precision calibration of threer dimensional light scanners. The invention specif- 
^ ica!1 y concerns a calibration object for calibration of all parameters of a three-dimensional light scanning system, the calibration 
object having at least one plane of symmetry and whereby at least one 3D object feature curve of each symmetric part is a continuous 
^ curve. The invention further relates to a method for calibration of a three-dimensional light scanner and a method for three-dimen- 
^ sional light scanning. Further specific embodiments for scanning of and modelling of objects for the ear and/or ear canal, and for 
^* scanning and modelling of dental implants are described. 
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Object and method for calibration of a three-dimensional light scanner. 
Technical field 

5 The invention relates to the field of precision calibration of three-dimensional light 
scanners. The invention specifically concerns a calibration object for calibration of all 
parameters of a three-dimensional light scanning system, a method for calibration of 
a three-dimensional light scanner and a method for three-dimensional light scan- 
ning. Furthermore, the invention relates to specific embodiments for scanning of and 
10 modelling of objects for the ear and/or ear canal, and for scanning and modelling of 
dental implants. 

Prior art 

15 System for three-dimensional light scanning are well known in the prior art. They 
typically comprise one or more light sources projecting a light sheet on the object to 
be scanned, one or more cameras and data processing equipment to convert the 
recorded image co-ordinates to three dimensional co-ordinates using state of the art 
software. The precision of the light sources and cameras are very high today as are 

20 also the software developed to detect the intersection of the light sheet with the ob- 
ject and convert the two-dimensional data to three dimensional co-ordinates. 
Therefore differences in and hence improvement of the precision primarily resides in 
the calibration of the systems. 

25 Precision is of utmost importance in the applications where the scan data are used 
for modelling an object which must fit precisely into or onto another object. Such 
applications are e.g. objects for the ear canal such as hearing aids, dental implants 
and other prostheses for the body. For hearing aid shells sub-millimetre precision is 
required or the shell will cause irritation and possibly infection to the epidermis of the 

30 ear canal. For dental implants the precision requirement is even greater, since a 
human being can detect differences less than 1/10 of a millimetre when biting. 
Therefore systematic or random errors in the calibration of scanners for these uses 
can be fatal. This has hitherto limited the use of scanning in the modelling of such 
implants and shells. 

35 
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Methods and object for calibration of three-dimensional light scanners are known 
from the prior art. The majority of such calibration methods are composed of several 
steps often using different calibration objects for calibrating different parameters. 
One general drawback is that the known methods do not allow an accurate calibra- 
5 tion of the light sheet offset. 

US 5,978,092 (BROWN) teaches a number of steps necessary for calibration of the 
system according to the publication. According to the disclosure several different 
objects of different shape are needed for calibration of the different parameters of 
10 the system. 

US 5,870,220 (REAL-TIME GEOMETRY CORP) also concerns the calibration of a 
scanner. Calibration for camera lens distortion is carried out by scanning a flat refer- 
ence surface with a repeated and evenly spaced pattern of laser-lines. As an alter- 
15 native an image of a rectilinear grid pattern on a flat reference surface can be 
scanned for calibration. Calibration of focal point value of the camera is carried out 
by scanning a flat reference surface on which an equilateral triangle of known di- 
mensions is drawn. It therefore appears that several steps and several calibration 
objects are needed to calibrate the scanner. 

20 

US 5,561,526 (LOCKHEED MISSILES & SPACE COMPANY INC.) concerns a cali- 
bration object and a method for calibration of a three dimensional scanner. The ob- 
ject shown in Figures 8, 9, and 10 is a plate with a plurality of substantially identical 
truncated pyramids each having countersunk pins During calibration the surface of 

25 the calibration object is first accurately measured by conventional traceable me- 
chanical methods. Thereafter the object is placed in the scanner and scanned. The 
intersections of the plane surfaces of the pyramids are calculated and the three di- 
mensional co-ordinates for the intersections are calculated. To this end approximate 
values of the system parameters of the scanner are used. Finally the calibration is 

30 carried out by varying the system parameters in order to minimise the difference 
between the mechanically measured co-ordinates and the estimated co-ordinates. 
This is done in a least squares sense. Calibration precision using this method is 
primarily determined by the accuracy of the initial mechanical measurement of the 
calibration object. The disclosed calibration object does not have a plane of symme- 



WO 02/16865 



3 



PCT/DK01/00564 



try to calibrate the laser sheet displacement and angle and furthermore has very few 
feature co-ordinates to use in the calibration. 

US 5,661,667 (VIRTEK VISION CORP) concerns a three-dimensional scanner 
5 wherein calibration is carried out by placing reflection targets on pre-determined 
locations on the object to be scanned so that the co-ordinates of the reflection tar- 
gets are known. The scanner can then be calibrated using these reference points. 

US 5,991,437 (REAL-TIME GEOMETRY CORPORATION) concerns a system for 

1 0 high accuracy calibration of a three-dimensional light scanning system, which pro- 
vides an object for calibration of a known geometry, such as a plane or a sphere. 
Upon that object is marked a pattern of points or geometric shapes, such as a set of 
equilateral triangles or their vertices. A set of computer programmed elements proc- 
ess photographic images of the object (containing a depiction of the pattern of geo- 

15 metric objects) and scanned images of the object to calibrate to high precision pa- 
rameters, including a focal point relationship value for scanning systems which use 
a camera as an image collector (which describes the relationships between the focal 
point of the lens and the light/image collection device of the camera), a set of values 
to correct for distortion of the lens, the relative distance between the image collector 

20 and the scanner light source and also an initial angle to determine light position 
scanning. The known geometric constraints of the object allow the location of the 
object to be determined by photographic images. By scanning the object and com- 
puting 3D X.Y.Z co-ordinates for those points against a constraint that the X.Y.Z co- 
ordinates fit onto the surface of the object, the system calibrates parameters per- 

25 taining to the relative distances between the image collector and light source, an 
initial angle of rotation for systems which employ movable light sources and other 
parameters. The system is applied to multiple camera scanning systems to calibrate 
rotation matrices and translation vectors which facilitate the joining of the scanned 
data. 

30 

Specific examples of pre-determined geometric shapes are one or several equilat- 
eral or right triangles. The calibration object may be a flat plane or a ball. According 
to the publication-there is no need to know the distance between the. different geo- 
metric shapes on the calibration object. Only the exact size of the geometric shapes 
35 need to be known. A disadvantage of such calibration is that the disclosed calibra- 
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tion object does not have a plane of symmetry to calibrate the laser sheet displace- 
ment and angle and furthermore has very few feature co-ordinates to use in the 
calibration. 



5 WO 98/59300 (REAL TIME GEOMETRY CORP.) concerns a system and method 
for modelling 3D objects and 2D images by wire frame mesh constructions having 
data points that combine both spatial data and surface information such as colour or 
texture data. The use of the complex data points (e.g., X, Y, Z, R, G, B in 3D and x, 
y, R, G, B in 2D) allows the modelling system to incorporate both the spatial features 
10 of the object or image as well as its colour or other surface features into the wire- 
frame mesh. According to one embodiment the system is adapted to determine the 
distance of each data point to a pre-selected reference object. Calibration is per- 
formed using a pre-selected reference object, which may be in the shape of a 
sphere or a plane. 

15 

WO 00/04506 (GEOMETRIX INC.) concerns a system and a method for computer 
modelling of 3D objects and 2D images by mesh constructions that incorporate non- 
spatial data such as colour or texture. 

20 According to one disclosed embodiment the system is automatically calibrated. The 
disclosed calibration object is of round shape. For calibration the disc shaped object 
of known size is secured to a turntable and it is viewed by the camera in an angle so 
that the image seen by the camera is of ellipse shape. The system subsequently 
determines the axes of the ellipse and calibrates the following parameters of the 

25 system 

a) the angle between the rotation axis of the disc and the ray connection the centre 
of the disc with the camera; 

b) the viewing angle subtended by a pair of image rays extending to the two end 
points of the minor axis; 

30 c) the angle of the image plane subtended by the axis of rotation of the disc target 
and the vertical image direction, and 
d) the distance and height of the camera. 

According to the disclosure it is not necessary for the camera to capture a view of 
35 the whole disc as long as one of the minor and both major axes can be seen. A 
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drawback of such a calibration method is primarily that very few data points are 
used and the points used do not cover the scanned area. Therefore the calibration 
precision is not great enough to allow for sub-millimetre calibration. 

5 US 6,044,170 (REAL-TIME GEOMETRY CORP.) concerns a system and a method 
for rapid shape digitising and adaptive mesh generation. According to the publica- 
tion the system can be calibrated using an auto-calibration procedure. During auto- 
calibration the camera scans a flat reference surface such as a wall covered by a 
white sheet of paper. Scanning of a two-dimensional object of known dimensions for 
10 calibration will not yield a satisfactory calibration for three-dimensional purposes. 

US 5,988,862 (CYRA TECHNOLOGIES INC.) concerns an integrated system for 
imaging and modelling three dimensional objects. In order to calibrate the lens dis- 
tortion, this distortion is removed by comparing the image the camera produces 

15 when aimed at a carefully designed and printed calibration target image. The differ- 
ence in the anticipated image and the recorded image provides the information 
needed to warp subsequently acquired images to eliminate the distortion. The publi- 
cation provides no details about the shape of the target image. Since the compari- 
son of the image and the printed image of the calibration target takes place in two 

20 dimensions - the image of the calibration target being printed - the method does not 
allow for precision calibration for three-dimensional purposes. 

US 5,999,840 (MASSACHUSETTS INSTITUTE OF TECHNOLOGY) concerns a 
method and a system for registration of three dimensional data sets, calibration of 

25 the laser scanning unit including the laser striping unit and the cameras is performed 
using scan data from a precisely machined shape referred to as a gauge. This 
known shape, along with the images of the laser scans from it, can be used to pre- 
cisely calibrate the laser instrument such that all subsequent measurements, any- 
where in the operating range of the scanning unit will result in accurate 3D meas- 

30 urements as measured relative to some fixed reference frame. The shape of the 
gauge is not disclosed. 

--- - US 6,081,273 (MICHIGAN STATE UNIVERSITY) concerns a method and a system 
for building three-dimensional object models such as virtual reality object models. 
35 Prior to scanning the system needs to be calibrated. This is done by scanning a 
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calibration reference object having a special 3D calibration pattern. The pattern 
shown as an example is a cube. For high accuracy, the 3D calibration pattern 
should be similar to the size of the object to be scanned. As an alternative to a cali- 
bration reference object, the object to be scanned may be used for calibration. This 
5 merely requires that the 3D co-ordinates of a number of reference points on the ob- 
ject are determined by measurement prior to calibration. 

Using the object to be scanned for calibration of the scanning system is a cumber- 
some way to perform the calibration, since measurement of reference points and 
10 calibration is required prior to every scan of a new object. Furthermore, since the 
disclosure provides no information as to where to select the reference points it is not 
possible to determine the precision of the calibration method. 

US 5,753,931 (NIKE INC.) features a method for capturing surface information re- 
15 garding the surface shape of an object, such as the underside of a human foot or 
other extremity. Also disclosed is a solid calibration gauge manufactured from alu- 
minium and having a sawtooth-type pattern. The surface of the calibration gauge 
comprises a plurality of nodes and ridges. The method for calibration includes com- 
parison of the "real" X, Y, and Z co-ordinates with the estimated co-ordinates and 
20 adjustment of the calibration parameters accordingly. The process is repeated until 
the difference between "real" and scanned co-ordinates is below a pre-determined 
threshold. 

US 5,549,476 (Stern) relates to a method of making dental restorations including 
25 temporarily repairing a tooth area to be restored to desired final shape, taking a first 
impression of the tooth area to be restored, preparing the tooth to be restored and 
forming a second impression within first impression with material that is non- 
adherent to the first impression material. The first impression and the second im- 
pression are separated. The second impression is scanned to obtain a digital image 
30 of the final restoration. A model of the dental restoration can the be made using 3-D 
milling. The publication concerns no information about calibration of the scanner. 

US 5,121,333, US 5,121,334, US 5,128,870, US 5,184,306, US 5,027,281, and US 
5,257,203 (REGENTS OF THE UNIVERSITY OF MINNESOTA) describe a method 
35 and apparatus for the automated reproduction of three dimensional objects of com- 
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plex and unique geometry. A computer acquires data describing an object and its 
surroundings, constructs a computer-based three dimensional model of the object 
from that data, superimposes an ideal geometry on the computer-based model, al- 
ters the ideal geometry to fit the form and function required of the reproduction, and 
5 then guides a milling machine in the fabrication of the reproduction. 

The publications primarily concern the manufacture of dental prostheses. It is men- 
tioned that the system can be used for the manufacture of in-the-ear hearing aid 
housings and moulds or implants to replace damaged bones but there are no ex- 
1 0 amples of such use. 

US 4,575,805 (MOREMANN ET AL) also concerns a method and an apparatus for 
the fabrication of custom-shaped implants. The surface characteristics of an organ 
that needs restoration, for example, a tooth which has been prepared for an inlay 
15 insertion, are read by means of a non-contact scan-head. The three-dimensional 
shape parameters of the implant required to restore the tooth in function and ap- 
pearance are computed on the basis of the recorded contour data. These parame- 
ters are then used in a program sequence which controls a milling, cutting or erosive 
process in order to manufacture the restorative inlay while the patient waits. 

20 

The publication further describes that the scanner can be calibrated by mapping a 
standard object, which has been measured utilising conventional techniques e.g. 
callipers. 

25 WO 00/34739 (FAGAN ET AL.) concerns a method for manufacturing hearing aid 
shells involving the use of a specially adapted ultrasonic probe head to safely 
measure the contours of the ear canal without contact with the surface being meas- 
ured. The recording of the data in the ear canal is made possible by filling the canal 
with a liquid and inserting the ultrasonic probe. The scan data are processed by a 

30 computer and the data are used with a rapid prototyping set-up such as stereo li- 
thography, selective laser sintering, laminate object modelling, inkjet modelling, 
fused depositing modelling, 3DP or any other system that produces real models 
from computer mathematical models to manufacture the hearing aid shell. One dis- 
advantage of the system is the need for filling the patient's ear completely with wa- 

35 ten This can extremely annoying to patients and they may experience symptoms of 
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nausea during and after such treatment. Furthermore it is doubtful whether it is 
really possible to determine the three dimensional co-ordinates of the surface of the 
ear canal with the required precision using an ultra sonic probe. An ultrasonic sound 
signal emitted does not travel in just one direction in the liquid in the ear canal. As 
5 the ultrasonic waves are reflected by the surface of the ear canal they travel in many 
different directions. Thus the transducers of the ultrasonic probe will detect a num- 
ber of signals of varying size with different delays after the emission of one sound 
signal. It must be extremely difficult for the underlying software to determine which 
of the signals to use in the determination of the co-ordinates. The disclosure pro- 
10 vides no information on how to perform this operation. 

EP 0 516 808 (T0PHOLM & WESTERMANN APS) concerns a method for computer 
assisted manufacture of otoplastic individually fitted to the contours of the ear canal. 
According to the described method a digital representation of the internal contours 
15 of the ear canal are used for milling or 3D printing of a hearing aid shell. A computer 
is used to optimise the location of the components of the hearing aid and the thick- 
ness of the walls of the shell. The disclosure does not suggest the solution of scan- 
ning the internal contours of the ear canal or a model of the ear canal using a laser 
scanner. 

20 

US 5,056,204 (ASCOM AUDIOSYS AG) concerns a method for milling of hearing 
aids whereby the internal contours of the ear canal are allegedly recorded by a laser 
apparatus located outside the ear of the patient. The disclosure contains no means 
to direct the laser light into the ear canal. Due to occlusion effects only part of the 
25 ear canal can be scanned according to the disclosed method. Furthermore, the dis- 
closure fails to take regard to the fact that the individual is likely to move while the 
image is recorded. 

Definitions 

30 

For the purposes of the present invention, the following definitions are provided. For 
a graphical representation of the terms, reference is made to the accompanying 
drawings especially Figure 22: 

35 3-D object feature curves 2201 
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3-D object feature curves are lines or curves on a calibration object The 3D object 
feature curves are characterised by real world 3D co-ordinates. When illuminated 
with a light contour such as that formed by a laser sheet, points on the 3D object 
5 feature curves are visible by a detection device such as a camera, a CCD or a 
CMOS. 3D object feature curves can be made by painting the lines on the calibra- 
tion object, so that they differ in colour from the other surface of the calibration ob- 
ject. In this case a colour sensitive detection device is required. More preferably they 
are edges formed on the calibration object, by e.g. milling away part of the object. In 
10 this case the feature points are detected as corners on the intersection contour cre- 
ated by the sheet of light on the object. 

2D- image feature curves 2202 

15 2D-image feature curves are created by a sheet of light on an object and are de- 
tected in the images recorded by the detection device (e.g. camera, CMOS, CCD). 
The detection comprises state of the art algorithms. One sheet of light may produce 
a number of 2D image feature curves. Where these multiple 2D image feature 
curves intersect image feature points are located having a 2D x and y image co- 

20 ordinate in the image in which it is recorded. The image feature points are repre- 
sentations of the 3D object feature curves. 

It is to be understood in the foregoing and in the following that when the term image 
is used, this also encompasses the term picture, which describes a 2-D image re- 
25 corded by a camera. 

Summary of the invention 

According to a first embodiment of the invention is provided a three dimensional 
30 calibration object having at least one plane of symmetry and whereby at least part of 
at least one 3D object feature curve of each symmetric part is a continuous curve. 
By providing the calibration object with a plane of symmetry, it becomes possible to 
calibrate all parameters, including the laser sheet parameters and the offset of an 
optional rotational angle. The continuous symmetric 3D object feature curves of the 
35 object assist in the calibration of the laser sheet parameters and angle offset by pro- 
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viding an exceptionally precise estimate for the plane of symmetry of the object. 
Furthermore, they provide numerous calibration points for use in the transformation 
of image co-ordinates to laser sheet co-ordinates and thereby to real-world co- 
ordinates. 

5 

Preferably the object is rigid, such as being manufactured from metal, preferably 
aluminium or from a material selected from the group consisting of alloy, stainless 
steel, a plastic polymer, kevlar®, carbon, wood. The rigidity of the object makes it 
more stable and the materials listed above all have excellent properties in respect of 
10 hardness, temperature expansion and workability. 

According to an especially preferred embodiment at least part of the surface of the 
object is rough or non-glossy. Ideally the surface of the calibration object should 
have the characteristics of a perfect lambertian surface to minimise the specular and 
15 diffuse reflection of light from its surface during calibration. 

The continuous curves of the at least part of the at least one 3D object feature curve 
are preferably described by continuous mathematical functions. Thereby the calcu- 
lation of points on the curves becomes especially easy, since this can be done 

20 merely by using the mathematical functions. This technical feature is also an ad- 
vantage if the object is milled using an automatic high precision milling machine, 
since the functions can simply be entered into the milling program. Preferably the 
functions describe part of a spiral, an ellipse, or a circle, or those that comprise 
trigonometric functions such as sinus, cosinus, tangens, cotangens. These mathe- 

25 matical functions have the advantage that the 3D object feature curves cover a large 
part of the camera field of view so that the calibration becomes precise over the 
whole field of view. 

Each symmetric part may have two, three, four, six or more 3D object feature curves 
30 being symmetric over the plane of symmetry. The presence of multiple 3D object 
feature curves improves the estimation of laser sheet parameters and angle offset. 
Preferably the distance between two 3D object feature curves of each symmetric 
part is constant, and more preferably all distances between 3D object feature curves 
of each symmetric part are constant. 

35 
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The object may have an axis of rotation such as an essentially vertical or horizontal 
axis of rotation. By recording numerous images of the object during rotation and/or 
translation the calibration performance is greatly improved. 

5 Improved calibration is obtained by having two axes of rotation or three, four, five or 
more axes of rotation. 

A mathematical combination of image feature points or features derived from these 
points plotted as a function of the angle of rotation q> or the translation preferably 

10 produces an estimable mathematical function. When this condition is fulfilled it be- 
comes easier to estimate the plane of symmetry of the object. Note that the image 
feature points are projections of 3D object feature curves and detected in images of 
the object recorded by a camera at discrete values of angle of rotation and/or trans- 
lation. The plotted mathematical combination may be the image feature point x co- 

15 ordinate, the image feature point y co-ordinate, the sum of the x and y co-ordinates, 
the product of the x and y co-ordinates, any other function of the x and y co- 
ordinates such as x" + y", x n - y n , y n - x n , x n * y n , x/y, y/x, x'Vy 0 , y7x n , sinus (x n + y n ), 
cosinus (x n + y"), tangens (x n + y n ), cotangens (x° + y"), sinus (x n * y n ), cosinus (x n * 
y n ), tangens (x n ♦ y"), cotangens (x n * y n ), exp (x n * y"), hyp (x n * y n ) or features de- 

20 rived from these points, e.g. the difference in rotation angle or translation between 
symmetric points. 

The estimable mathematical function obtained by the plot of the mathematical com- 
bination of image feature points or derived features as a function of rotational angle 
25 or translation preferably can be modelled with a first order approximation, a second 
order approximation, a third, fourth, fifth, sixth, seventh, eighth, ninth, tenth, or n m 
order approximation. These functions are easily estimated using state of the art 
mathematical software. 

30 Preferably the object has a size corresponding to the size of the objects to be 
scanned later. Thereby a precise calibration of the three-dimensional room housing 
the object to be scanned later is obtained. The linear size of the object could be 
between 50 and 150 % of the linear size of the object to be scanned later, such as 
between 60 and 140 %, for example between 70 and 130 %, such as between 75 

35 and 125 %, such as between 80 and 120 %. 
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The object may also have a size corresponding to the camera field of view. Thereby 
the whole field of view can be calibrated and irregularities in e.g. the lens can be 
compensated for during calibration. The linear size of the object may be at least 
5 10% of the camera field of view, such as at least 20%, for example at least 30%, 
such as at least 40%, for example at least 50%, such as at least 60%, for example 
at least 70%. such as at least 75%, for example at least 80%, such as at least 85%, 
for example at least 90%, such as at least 95%. 

10 For certain applications, such as for scanners for ear scanning it is advantageous 
that the calibration object is hollow and has the 3D object feature curves on the in- 
side. 

To provide improved precision, the calibration object may be provided with fastening 
1 5 means to secure it to the scanner or the table of the scanner. 

According to a second aspect of the invention is provided a method for calibration of 
a three dimensional light scanner comprising the steps of providing a three dimen- 
sional light scanner, providing a calibration object according to the invention, scan- 

20 ning the calibration object, determining image feature co-ordinates being represen- 
tations of at least one pair of 3D object feature curves for each of a discrete number 
of values of an angle of rotation and/or a translation, a pair consisting of one 3D 
object feature curve in each symmetric part of the calibration object, changing the 
calibration parameters to fit the calibration object. Due to the presence of the 3D 

25 object feature curves and plane of symmetry, a very accurate estimation of the laser 
sheet parameters can be obtained in one calibration step. 

Preferably, a sheet of laser light is projected onto the calibration object to produce 
image feature curves. 

30 

When preferred, the image feature lines may also be determined using the Hough 
transformation, filter search, max intensity, threshold, centre of gravity, derivatives or 
other procedures. 
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The image feature co-ordinates are found as the intersection between image feature 
curves. These intersections could be seen in the images as corners or sharp edges 
of the image feature curves. The image feature co-ordinates may be found as the 
intersection between image feature curves such as the intersection between two n" 1 
5 order curves, as the intersection between two first order curves, as the intersection 
between two second order curves, as the intersection between two third order 
curves, as the intersection between a fist order curve and a second order curve, as 
the intersection between a first order curve and a third order curve, or as the inter- 
section between a second order curve and a third order curve or as the intersection 
10 between any other possible combination of curves. 

Preferably, the calibration method further comprises plotting of a mathematical com- 
bination of image feature points or features derived from these points against the 
angle of rotation q> or the translation of the calibration object. By plotting this function 
15 and optionally estimating a mathematical function describing the relationship be- 
tween the function of an image co-ordinate and the angle of rotation, estimation of 
the laser sheet parameters and angle of rotation becomes especially precise. The 
method may further comprise determination of the mean plane of symmetry in the 
plot. 

20 

The mean plane of symmetry can be determined by calculating the mean angle of 
rotation / mean translation for pairs of image feature points having the same value in 
the mathematical combination. Doing this produces multiple estimates for the en- 
coder offset and laser sheet displacement allowing also for the estimate of the laser 
25 sheet angle. 

Alternatively mathematical formulas can be derived for the curves, which appear in 
some of the plots of the mathematical combination as a function of the angle of rota- 
tion or the translation. Given these curves and the corresponding formulas the en- 
30 coder offset, the laser sheet displacement and the laser sheet angle can be esti- 
mated. 

Preferably laser sheet co-ordinates of the 3D object feature curves are estimated 
corresponding to a discrete number of values of angle of rotation and/or translations. 
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These values should preferably cover the whole circumference and/or length of the 
calibration object. 

2D co-ordinates of the 3D object feature curves corresponding to a discrete number 
5 of values of angle of rotation and/or translation may be calculated from mathemati- 
cal functions determining the 3D object feature curves. In order to determine the 
calibration parameters such as camera position, camera orientation, and camera 
optic parameters, pairs of 2D laser sheet co-ordinates are compared to calculated 
2D co-ordinates for a discrete number of values of angle of rotation. This compari- 
10 son preferably comprises using the Tsai or the Heikkila algorithm. The advantage of 
the Tsai and the Heikkila algorithm in this context is that it provides rapid and pre- 
cise estimation of the calibration parameters such as radial lens distortion. 

Alternative methods for calibration comprise direct linear transformation, and direct 
15 non-linear matrix transformation optionally in combination with an optimisation pro- 
cedure such as least squares means to minimise the error. In these cases initial 
calibration parameters may be estimated to facilitate the convergence of the pa- 
rameters during optimisation. 

20 To improve calibration precision outliers may be excluded from the calibration. Out- 
liers can e.g. be identified in the plot of the mathematical combination of image fea- 
ture co-ordinates against the angle of rotation / the translation or by back projection 
of co-ordinates after an initial calibration. 

25 2% of the feature points deviating most from the back-projected 2D image feature 
curves may be excluded from the calibration or at least 3%, such as at least 5%, for 
example at least 10%, for example at least 12%, such as at least 15% for example 
at least 20, preferably at least 25%, for example at least 30 %, more preferably at 
least 33 % may be excluded to improve calibration precision. 

30 

In order to cover the whole circumference and/or length of the calibration object the 
discrete number of values for angle of rotation / translation may be at least 100, 
preferably at least 240, for example at least 500, such as atjeast 750, for example 
at least 1000, such as at least 1200, for example at least 1500, such as at least 
35 1800, for example at least 2000, such as at least 2400, for example at least 3000, 
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for example at least 3600, such as at least 4200. The higher the discrete number of 
values of angle of rotation / translation, the higher the calibration precision. 

According to a third aspect of the invention is provided a method for three- 
5 dimensional scanning comprising the steps of providing a three-dimensional light 
scanner, calibrating the system, scanning an object determining image co-ordinates 
on the light contour for each of a discrete number of values of an angle of rotation/ a 
translation, calculating three dimensional real world co-ordinates describing the 
surface of the object. The advantages of a scanning method according to the inven- 
10 tion is the improved precision compared to prior art scanning methods. The preci- 
sion normally lies in the sub-millimetre range. 

Preferably calibration is performed using a calibration object according to the inven- 
tion using the calibration method according to the invention. 

15 

The three-dimensional real world co-ordinates may be calculated from the image co- 
ordinates using the Tsai algorithm, the Heikkila algorithm, direct linear transforma- 
tion or be estimated using direct non-linear matrix transformation. Image co- 
ordinates may be determined using Hough transformation, filter search, max inten- 
20 sity, threshold, centre of gravity, and derivatives or similar known algorithms. 

The three-dimensional light scanner preferably comprises at least one light sources 
and at least one cameras such as at least two light sources and at least two cam- 
eras such as at least three light sources and at least three cameras, for example 
25 three light sources and six cameras. By increasing the number of cameras and light 
sources, occlusion effects are minimised and a more precise scanning of the whole 
three-dimensional room housing the object to be scanned is obtained. 

The object to be scanned may be rotated and/or translated during scanning or the 
30 camera and the light source may be rotated and/or translated around the object 
during scanning. The actual motion may advantageously be planned by an auto- 
matic motion planner, which creates the optimal motion for the individual object, 
camera and light source. One advantage associated with . this embodiment is that 
occlusion effects can be minimised and in some cases eliminated. 

35 
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The object to be scanned may advantageously be aligned manually to the centre of 
a rotating plate using the cross section between two laser sheets as a guiding refer- 
ence. Thereby very precise placement of the object in the centre is possible. It is not 
possible to centre the object using a marking on the rotation plate, since this mark- 
5 ing will be covered by the object. 

More preferably, the object is mounted on e.g. a spike, which may be located at the 
rotational centre. This elevates the object and minimises occlusion effects caused 
by the rotating plate. This embodiment is especially advantageous for small objects 
1 0 and objects where the whole surface needs to be scanned. 

Preferably the light source comprises laser light and/or structured light. The camera 
used for the scanner may be a video camera, a CCD or a CMOS. 

15 In order to cover the whole circumference and/or length of the object the discrete 
number of values for angle of rotation / translation may be at least 100, preferably at 
least 240, for example at least 500, such as at least 750, for example at least 1000, 
such as at least 1200, for example at least 1500, such as at least 1800, for example 
at least 2000, such as at least 2400, for example at least 3000, for example at least 

20 3600, such as at least 4200. The higher the discrete number of values of angle of 
rotation / translation, the higher the scanning precision. 

According to a fourth embodiment of the invention is provided a method for manu- 
facturing a shell comprising a device for the ear of an individual comprising the steps 

25 of providing three dimensional co-ordinates describing the surface of the ear canal 
using three-dimensional scanning, and subsequently modelling a three-dimensional 
model. The ear and ear canal differ from individual to individual and therefore a pre- 
cise measurement of each person's ear is required to get a satisfactory fit of a de- 
vice to be carried in the ear. Therefore, a measurement of the dimensions of each 

30 individual's ear and ear canal is required. The advantage over prior art techniques 
for modelling devices for the ear is that the production time is significantly reduced 
combined with an improvement in the quality. 
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The data describing the dimensions of the individual's ear and/or ear canal may be 
provided by scanning a model of the ear canal of an individual. To improve the 
scanning precision the surface of the model preferably is non-glossy or rough. 

5 The data are more preferably provided by scanning the ear canal of an individual. 
Thereby, the steps for manufacturing the device are reduced to the absolute mini- 
mum and an especially perfect fit of the device can be obtained. The ear and/or ear 
canal may be scanned a number of times, such as under different conditions affect- 
ing the geometry of the ear canal of the individual. Thereby the variations in the di- 
10 mensions of the ear and the ear canal can be recorded. This is impossible using the 
prior art techniques. Methods and apparatus for scanning the ear canal are dis- 
closed in PCT/DK01/ (Method and apparatus for three-dimensional optical 
scanning of interior surfaces, 3-Shape) filed on 24. August 2001 . 

15 When the variations in the ear canal have been recorded, computer modelling of the 
device for the ear by adjusting the model for the dynamic changes in the geometry 
of the ear canal using scan data from different scans can be carried out and a de- 
vice which will fit under all circumstances as a compromise between the varying 
dimensions of the ear and ear canal. 

20 

Especially precise scanning is obtained when carried out according to the method 
described above. 

Once the data are recorded the device may be manufactured using any automatic 
25 manufacturing technique such as milling. More preferably the modelling technique 
comprises 3-dimensional printing, stereo lithography, selective laser sintering, lami- 
nated object modelling, inkjet modelling, fused deposition modelling, nano-printing. 
Common for these techniques is that only the required amount of material is used 
and that it is easier to model complicated models such as devices for the ear and/or 
30 ear canal. 

The device may comprises a hearing aid, a mobile phone, a loud speaker, a micro- 
phone, communication devices, a tinnitus masker or a tinnitus masking device such 
as the ones described and US 5,325,872, WO 91/17638. 
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The invention also encompasses the devices manufactured according to the method 
described. 



The invention furthermore features a method for manufacturing a dental implant for 
5 an individual comprising the steps of providing three dimensional co-ordinates de- 
scribing the surface of the dental implant using three-dimensional scanning, model- 
ling a three-dimensional dental implant. 

The data describing the surface of the dental implant may be provided by scanning 
10 a model of the dental implant for the individual, the model preferably being non- 
glossy or rough. But the data can also be provided by scanning the tooth in situ in 
the patient. In addition data from a tooth scanned in situ can be combined with data 
obtained from a scan of the surface to which the dental implant is to be attached. 

15 The dental implant can be manufactured according to any of the methods described 
above for the ear device using different materials. Preferably the data describing the 
surface of the dental implant are obtained by scanning according to the method de- 
scribed above. 

20 Brief figure description 

Figure 1 illustrates a CAD model of a scanner according to the invention. 
Figure 2 shows a camera image of the setup shown in Figure 1 . 

25 

Figure 3 illustrates an embodiment of the scanner with two cameras, one laser and 
a rotation plate. 

Figure 4 illustrates an embodiment of the scanner with four cameras, two lasers and 
30 a rotation plate. 

Figure 5 illustrates an embodiment of the scanner with six cameras, three lasers and 
a rotation plate. 
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Figure 6 show how occlusion effects can be reduced by placing the object on a 
spike. 

Figure 7 illustrates a laser source projecting multiple laser sheets on the rotational 
5 plate. 

Figure 8 shows two laser sources projecting laser sheets on the rotational table to 
produce a cross section. The cross section can be used to guide the placement of 
the scan object. 

10 

Figure 9 illustrates an embodiment of the scanner with six cameras, three lasers, a 
rotation plate and a linear axis. 

Figure 10 shows the object movement during shutter and image transfer. 

15 

Figure 1 1 shows a CAD model of the rotating plate mounting system. 

Figure 12 illustrates the specular, diffuse specular and uniform diffuse reflections 
from the object surface. 

20 

Figure 13 shows the relation between Hough and image space. 

Figure 14 shows an image of an object intersecting the laser sheet (left) and a num- 
ber of negative images of the contour (right). 

25 

Figure 15 illustrates the results of transformation in Hough space. 

Figure 16 gives an overview of the co-ordinate system for which a transformation 
needs to be found and calibrated. 

30 

Figure 17 show a possible configuration of a calibration object. 

Figure 18 illustrates the laser sheet co-ordinate system relative to the object co- 
ordinate system, (a) the definition of q>=0 (encoder position=0). (b) rotation relative to 
35 the zero angle. 
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Figure 19 shows three CAD models of different calibration objects. 1901 prior art 
calibration object. 1902. object according to the invention. 1903 preferred embodi- 
ment of the calibration object according to the invention. 

5 

Figure 20 shows a CAD model of a preferred calibration object according to the in- 
vention as seen from the camera. 

Figure 21 illustrates a graphic representation of eight 3D object feature curves of a 
10 preferred calibration object. 

Figure 22 shows an image of the milled calibration object from camera 1. 

Figure 23 shows two images (negative) captured from the calibration program. 

15 

Figure 24 illustrates two screen dumps from the calibration point processor program, 
(a) raw corner detected during one full rotation, (b) selected 2D image feature 
curves corresponding to the calibration object 3D object feature curves. 

20 Figure 25 shows X-co-ordinates of extracted corner features plotted as a function of 
the encoder position (proportional to the rotation angle <p). 

Figure 26 shows identical feature location in image space. 

25 Figure 27 illustrates the extraction of encoder offset and laser sheet parameters 
from encoder mean position of symmetric image features. 

Figure 28 illustrates the geometrical relations used for estimation of the laser sheet 
parameters, d - laser sheet displacement, <p 0 - encoder offset, a - laser sheet angle. 

30 

Figure 29 shows three camera images of an ear impression. 

Figure 30 shows the result of scanning of the ear impression in figure 29. 
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Figure 31 illustrates a scanner according to the invention adapted for scanning of 
the ear and ear canal. 

Figure 32 illustrates another embodiment of the scanner for the ear and ear canal. 

5 

Figure 33 shows an embodiment of the ear scanner, where a lens system is used as 
image guide. 

Figure 34 shows an embodiment of the ear scanner, where fiber optics are used as 
10 image guide. 

Figure 35 shows an embodiment of the ear scanner, where the camera is placed on 
the head of the scanner. 

1 5 Detailed description of the invention 

In the following a description of the preferred embodiment is provided. 
Mechanical construction 

The mechanical construction shown in Figure 1 was designed using SolidWorks, a 
20 modern CAD modelling tool. Most of the mechanical parts have been customised 
exclusively to fulfil the requirements of the scanner. The actual milling was done 
partly manual and partly using MasterCam, a modern CAM (Computer Aided 
Manufacturing) system. 

25 The laser scanner of Figure 1 comprises a light source 101, a camera 102) a rota- 
tional plate 103) and a frame 104. During scanning and calibration, the object to be 
scanned is placed and secured to the rotational plate, a sheet of laser light is pro- 
jected onto the surface of the object and images are recorded by the camera as the 
object moves. 

30 

In Figure 2 is shown a photograph of the built laser scanner with a preferred calibra- 
tion object 201 in place on the rotational table. 
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The following series of figures illustrate variations in the laser scanner hardware. 
Figure 3 shows an embodiment with two cameras 301 positioned on either side of 
the laser source 302 above the rotational table 303. In Figure 4 is shown an em- 
bodiment with four cameras 401, 401", 401", and 401'" together with two laser 
5 sources 402. 402*. With this setup occlusion effect are minimised. Even better mini- 
misation of occlusion is obtained with the setup in Figure 5 with two series of each 
three cameras 501 located on either side of a series of three laser sources 502. A 
number of extra bars 503 have also been added to increase the flexibility and stabil- 
ity of »>e construction. Another way of reducing the occlusion effects of the scan 

10 objet itsetf and of the rotational plate is shown in Figure 6, where the scan object 
601 is located on a spike 602 or similar means above the rotational plate 603. Two 
laser sources 604 project laser sheets on the object 601 and images are recorded 
by four cameras 605. In Figure 7 a laser source projecting multiple laser sheets 701 
on the rotational plate are shown. With this setup more effective scanning is ob- 

15 tained since more features can be extracted at any one time. In Figure 8 two laser 
sources project laser sheets on the rotational table to produce a cross section. The 
cross section can be used to guide the placement of the scan object or calibration 
object on the rotational plate. Figure 9 shows an embodiment with both a rotation 
901 and a translation axis 902. This embodiment makes it possible to rotate as well 

20 as translate the object with respect to the camera. The construction can easily be 
expanded to include two, three, four, five, six or more rotation and/or translation 
axis. 

The construction is based on a desire for flexibility-, size- and stability-constraints. 

25 Though the actual size is not considered important, scalability of the developed and 
adopted solutions have been paid attention to, in order to enable scale up of the 
complete system. The diameter of the rotation plate was chosen to be 120 mm, 
enabling a relatively small and handy construction. Another reason for which to keep 
the construction as small as possible was to enable customised parts to be created 

30 with a customary milling machine (medium size CNC milling machine and turning 
lathe). 

The absolute precision of the CNC machine (OKU MA O700) is assumed to be better 
than 0.02 mm (based on the milling machine parameter setup), whereas the relative 
35 precision is assumed to be better than 0.01 mm, depending on the desired milling 
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speed. Some parts have been latched, and then milled (e.g. the rotating plate), re- 
quiring a "manual" origin calibration between the two operations. This will obviously 
introduce some extra inaccuracy to the created object. 

5 Figure 1 to Figure 9 show the hardware construction, which is designed to be as 
flexible as possible, yet maintaining a stable and rugged design. The cameras can 
be translated and rotated in all direction in order to optimise the FOV (Field of view), 
focus, depth of field, etc. Furthermore, a special laser-mounting device has been 
developed to allow fine adjustment to the laser sheet position. Another important 
10 issue, paid special attention during the design phase, was stability of the construc- 
tion. Connectors and mounting devices was designed and made in hardened steel 
and high quality aluminium. All manually movable parts are designed with multiple 
bolt fixtures enabling a stable and noise immune static setup. 

1 5 Test of mechanical stability 

To test the stability of the construction a jx-watch was attached to one of the cam- 
eras, and selected features where manually bent or forced to move. 

It was possible to change the relative camera position approximately +/-0.1 mm, by 
20 applying a somewhat large manual force (approximately 10-20 kg). Fortunately the 
camera stabilises to its original position (within the resolution of the watch, 2fim = 
0.002 mm) as the force is removed, and the basis mechanical construction was 
therefore considered sufficiently accurate and stable. Moreover, if a more stable 
steel construction is required, it will increase the weight further (with the disclosed 
25 construction the total weight is approximately 50 kg). 

The lens itself was also tested for mechanical stability. It was found that it could be 
moved +/-0.2 mm in all directions, if the lens fixtures have not been tightened. As 
discussed later it is therefore necessary to fix both focus and lens aperture. 

30 

Temperature expansion 

The linear temperature expansion at 25 °C is approximately 23-1 0" 6 K" 1 for alumin- 
ium and 6-10 -6 K~ 1 for the used steel (i.e. one meter steel expands approximately 
0.006mm pr. °C). Approximately 90% of the construction is made of steel, thus the 
35 combined expansion is assumed to be approximately 7-10" 6 K"\ The distance be- 
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tween the laser and the camera in the implemented setup is approximately 0.16 m, 
resulting in 0.001mm displacement pr. K. The "sfeef distance from the rotating plate 
to the camera is approximately 0.6 m, resulting in a maximal displacement of 0.004 
mm/K. Using the scanner at normal conditions the temperature will only vary a few 
5 Kelvin and the expansion is not considered to be a problem. 

Laser 

The following factors constitutes to the choice of laser: 



• Lasers compatible with line generating optics. 

10 • Visible by the camera, (i.e. The wavelength of the laser must be sensible by 

the camera) 

• Line width and intensity distribution. 

• Depth of field, focus range within the limits of the construction. 

• Focus adjustable high quality laser. 

15 

The laser chosen was a 10mW laser from Lasiris. The laser wavelength 635 nm 
(smallest possible) was chosen to obtain optimal camera sensitivity. At 630 nm the 
relative camera CCD sensitivity is 65% (65% relative camera sensitivity at 630nm, 
according to the camera specification sheet provided by JAI ). The laser is mounted 

20 with a pattern head generating a laser line with a fan angle (FA) of 20 degrees. The 
distribution along the laser line is linear whereas the light distribution across the la- 
ser line is Gaussian. The laser was mounted on a customised CAD modelled de- 
sign, allowing the laser to be moved, in order to change the distance and angle to 
the scanned object. By moving the laser, the focus and depth of field can be opti- 

25 mised. 

The line thickness is measured at the 1/e 2 point of the Gaussian distribution. Depth 
of field is defined as twice the Rayleigh range, the distance at which the line thick- 
ness has increased to V2 times of its minimum width. 



30 



In the implemented setup the object distance can be varied between 300 and 500 
mm, consequently the minimal laser thickness will be approximately 0.2 mm. The 
laser thickness will limit the possibility of scanning small details. However, it will be 
Gaussian distributed on continuous surfaces, and it is possible to estimate the peak 
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with an accuracy much higher than 0.2 mm. In another example, a laser source was 
used, which provided a laser thickness of approximately 0.09 mm for improved pre- 
cision. 

5 The depth of field corresponding to object range in the implemented setup is ap- 
proximately 200 mm, sufficiently large to cover the scan area. 

The (an angle (FA) determined by the chosen pattern head, must be large enough to 
cover the desired scan area. The laser angle (LA) can be adjusted manually to 
10 minimise occlusion effects. 

The CAD modelled laser mounting system was design to enable fine adjustment of 
the laser line position at the object being scanned. The laser line forms a laser 
sheet, ideally equal the ZY-plane (zero rotational angle), featuring a simple trans- 

15 form of coordinates. The screws are chosen very small (03 mm) to obtain a fine 
pitch, thereby enabling accurate adjustment capabilities. However it is practically 
impossible to adjust the laser sheet to exactly equal the ZY-plane. This is obvious 
considering the following: If the object range is 500mm the relative screw adjustment 
distance is 10 times smaller (0/2=5Omm). A small adjustment (e.g. 0.01 mm) of the 

20 laser position will thus result in a 10 times larger (0.1mm) movement of the laser 
sheet at object range. Furthermore it is hard to tell whether the sheet is right posi- 
tioned using manual inspection. Therefor a calibration of the laser sheet is required. 

Camera and grabber 

25 

Choice of camera 

The choice of camera(s) is obviously crucial to the performance of the scanner, and 
the following key parameters constitutes to the choice of camera: 

30 • Progressive scan and asynchronous reset, featuring triggered grabs. 

• Cost, resolution, frame rate and sensitivity. 

• General quality (e.g. S/N-ratio) and ability to interface with a standard grab- 
ber card. 
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To limit the number of options and optimise cost-benefit, a monochrome camera 
solution was chosen. 

JAI Ml, monochrome high resolution camera 

It was found that the only progressive scan camera featuring high resolution and 
5 frame rate at a relatively low cost was CV-M1 , a high performance camera from JAI. 
The camera is based on a 2/3-inch progressive scan CCD image sensor from 
SONY. The monochrome (8-bit) censor has a resolution of 1300x1030 pixel ele- 
ments and it features approximately 12 frames pr. second in continuous mode. In 
asynchronous reset mode the frame rate is somewhat lower, depending on the 
10 shutter time. In the adopted setup the optimal shutter time was found to be 1/24 
second. In asynchronous reset mode the frame is transferred to the grabber in a 
1/12 seconds period succeeding the shutter time, resulting in a maximum frame rate 
of approximately 24/3=8 frames pr. second. 

15 The chosen grabber card was a Matrox Meteor ll/MC (Multi Channel), likewise sup- 
plied by JAI. The grabber was chosen based on its ability to handle the large 
amount of image data, and the support of asynchronous triggering. The setup 
adopted, features computer controlled asynchronous triggered grabs from one of the 
two software selectable cameras. 

20 

Camera mounting system 

The mechanical camera mounting system was designed to enable manual transla- 
tions and rotations. The system is thereby made flexible in terms of scan area and it 
is possible to adjust the cameras to a specific assignment (featuring a special object 
25 geometry), thereby minimising occlusion effects, and maximising the accuracy. Be- 
sides the dynamic requirements, it was designed to have a maximal static stability, 
and immunity to external forces and noise. This was reached using hardened steel 
and multiple bolt fixtures. 

Lens 

30 Another crucial part of the vision system is the camera lens. The following basic 
demands, concerning the lens, are important in order to obtain the highest possible 
image quality within the scope of the construction: 
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• Appropriate FOV (field of view). To obtain the highest static resolution the 
FOV should only cover the area of interest (the scan area). 

• Optimal resolution and sharpness 

• Standard (off-the-shelf) lens 

5 • Focus at short range (approximately 0.2 mm), to minimise the size of the 

construction and enable a customised short-range setup. 

• Large DOF (depth of field). To obtain sharp images a large DOF is needed. 

• Appropriate focal length. In order to minimise lens distortion, the focal length 
should not be very small. 

10 • C-Mount, compatible with the cameras. 



Based on the initial construction setup and the demands above, a number of lens 
restriction parameters can be listed. Some of the parameters are determined by the 
camera and cannot be changed, others are rough estimates based on the initial 
15 setup and therefore changeable within a certain range. 





• Wfov 


« 120 mm 


(Initial scan area with) 




• Hfov 


* 100 mm 


(Initial scan area height) 




• lo 


e [300; 400] mm 


(distance from the lens to the object) 


20 


• k 


* 11-22 


(aperture stop, for which the laser is visible) 




• Wee 


= 8.7 mm 


(CCD width) 




• Hoed 


= 6.9 mm 


(CCD height) 




• Wp ixe , 


= 6.7 \im 


(CCD pixel element size) 




• f 


e[16; 50] 


(focal length, restricted to standard lenses) 


25 


• X 


« 635 nm 


(wave length of the laser light) 



Tilting the cameras might optimise W F ov and H F ov. 



Field of view 

30 The object distance /o, the image plane distance li, and the focal length f are con- 
nected with the FOV and the CCD size, by the lens low, and the magnification ratio 
(m) PL 
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FOV 



CCD 



lo lo 



10 



Depth of field 

In order to obtain optimal quality of the laser contour, the captured images have to 
be as sharp as possible. The depth range of which the image must be sharp is de- 
pendent on many factors, mainly the scan area, the laser sheet position and the 
relative laser-camera angle. Ideally the intersection between the scan area and the 
laser sheet equals the FOV, and the image must be sharp in that area. A sharp im- 
age is obtained with the right focus and a large DOF (depth of field). Where DOF is 
the depth range in which the image is not blurred more than a maximum defined 
limit [2]. 



Where 6 is the diameter of the circle of confusion or blur measured on the CCD 
plane. / is the distance to the plane of interest (i.e. the most focussed, or sharpest 




(lo-f)k 



plane), and k = 



. DOF is defined as the range of / given a constant circle of 



aperture 



confusion with diameter 5. 
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8 < constant 



lo 



-1 



lo 



lo 



lo-lx, 



-1 



< constant 
lo 



lo + lx 0 



-1 



, where |/x, -lx 2 \ = DOF 



6 = 



\\lo-DOFI2 ) {lo + DC 



DOF 1 2 



— 1 



/2x / 2 



5 _ f 2 DOF 
2lo(lo-f)k 



(lo-f)k 



f 2 



2lo DOF 



K DOF 2lo J 



\lo-f)k 



DOF = 



2Slo(lo-f)k 
f 2 
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Choice of lens 

Due to the fact that DOF is proportional to the aperture stop (k), it is preferable to 
have a large aperture stop (16-22). The drawback of a large aperture stop is that the 
amount of light passing trough the lens is reduced. To compensate for this, it might 
be necessary to increase the shutter time of the camera, thus introducing a new 
drawback due to object movement during the shutter time. In figure 10 object 
movement during shutter (t1, typically 1/24 second) and image transfer (t2, 1/12 
second) as a function of angular speed and scan area size are shown. In the imple- 
mented setup, the typical magnitude of x1 is 0.02 mm, the typical magnitude of x2 is 
0.05 mm to 1 .5 mm depending on the rotational speed. 



15 To find the optimal choice of lens the following table was made. 



5max= n pixel = n x 6.7 um 



lo [mm] 


f [mmj 


k (aperture stop) 


DOF [mm] 


300 . '; 


21.8 




nx27 


350 ; . .. ; 


25.4 


16 :■ V.. 


n x 39 


400 = 


29.0 


22 't. ■ -• 


n x 54 


450 :-; 


32.6 


16 


n x 39 


500 


36.3 


16 • " - 


n x 39 
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The object distance in the initial implemented setup was 300 - 400 mm, thus a lens 
with 25 mm focal length will be suitable. A 25mm standard lens from Pentax was 
therefore chosen. 

5 

Changing focus and aperture 

If the camera is moved and the relative object distance thereby changes, the lens 
focus will obviously have to be updated, and thus the whole scanner system must 
be re-calibrated. If only the aperture stop is changed (e.g. when scanning objects 

10 reflecting the laser light differently), it might not be necessary to recalibrate. How- 
ever it was found that a manual adjustment of the aperture, might well cause the 
imaged frame to be misaligned or moved more than 1 / 2 pixel. Although it might not 
seem to be a large error source, it will actually result in an error of more than 0.04 
mm on the final scan result. To prevent the error, focus and aperture stop was 

15 locked with the provided bolt fixtures. The drawback of fixing the aperture is that it 
will not be possible to scan a bright object (e.g. white) using the same setup as for 
scanning a black light absorbing object. To solve the problem the laser illumination 
was made adjustable. 

20 Motor, encoder and gear 

To enable a precise calculation of the laser sheet position relative to the object co- 
ordinate system the rotation angle of the round plate must be known with high accu- 
racy. Furthermore the rotation angle should be steady during the camera shutter 
time to avoid multiple feature detection, and blurred laser contour. It was considered 
25 to buy a reassembled high precision rotary table, but due to high pricing and unnec- 
essary complexity, it was decided to design and create a customised solution. First 
of all a suitable motor is needed to enable the actual motion, secondly the angular 
position must be available. Two solutions were considered: 

30 • Stepper motor with gear. 

• Linear motor with gear and encoder. 

A linear motor solution was chosen for the following reasons: 
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• Possibility of high speed scanning (pre-scan). 

• Accurate and absolute position readout, without the risk of "loosing 0 steps. 

• Limited demands to the controlling system. 

• Smooth nonstop motion, preventing gear and motor backlash problems. 

5 

Nonstop movement 

To prevent backslash in the gear and faulty position readout it was decided to let the 
motor run at constant speed during the scanning. Furthermore the smooth motion 
enables a constant synchronism between the captured image and the angular posi- 
10 tion recorded. The drawback of continuous motion is that the object moves during 
the camera shutter time, thus small errors proportional to the angular speed might 
appear. The geometric relations shown in Figure 10 and below contribute to the final 
choice of motor speed, gear ratio and encoder resolution. 



15 



25 



<t> m otor=*>plate Gear Rati0 



J "encoder = *>mo<or ' EfiCOdeT ties P T. revolution 



30 Table 1 describes the motor, gear and encoder that were chosen. The components 
were supplied by LMT-Transmission and Danbit: 



Component 


Name / Vendor 


Key-parameters 








Motor 


Penta 5X / Motor Power 
Company 


Linear DC-motor, Speed= 100-3000 rpm, 
Operating toque=0.32 Nm 


Gear 1 


Reg 42/ Eisele 


Precision planetary gear. Ratio=1:30. 
Straight. 


Gear 2 


BW-W / Brown Group 


Precision planetary gear. Ratio=1:30. 90 
degree. 


Encoder 


2RHI / Scancon 


Incremental (A,B) output. Max speed=3000, 
400 tics pr. round, resulting on 352,000 
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encoder pulses pr. rotation of the rotational 
plate. 








Motor con- 
trol 


400 / Sprint Electric 


One quadrant one phase linear DC motor 
speed controller, (i.e. only one way). Speed 
proportional to input voltage (0-10 voltage) 


Encoder 
card 


PLC833 / Advantech 


Encoder tics to absolute position converter. 
24 bit resolution. 


DAC 


AD16S12/DanBit 


Digital to analogue converter, used to con- 
trol the motor speed. 



Table 1: Components chosen to control the rotation of the plate. 



The choice of encoder and gear ratio leads to the following angular resolution on the 
rotation plate: 

5 

Angular resolution = 

360 degrees = 360 degrees = QQQ j degree 

Encoder tics pr. round x gear ratio 400 x 30 x 30 

The linear resolution on the plate movement 50 mm from the centre axis, is conse- 
quently: 

Linear movement 50 mm from center axis = 

1 0 tcxD = TcxlOO mm = qqqq^j mm = 0.87pm 

tics/round 360000 tics/round 

The actual angular resolution may well be less accurate than the theoretical resolu- 
tion, due to non-linearity in the gear and encoder. 

15 Object movement during shutter and image transfer, with the chosen motor and 
gear 

The object movement (x1 and x2) 50 mm from the centre axis during the shutter , 
time (t1) and the image transfer time (t2), was calculated with slow (100 rpm) and 
fast (3000 rpm) motor speed (see Figure 10). In addition the scanning time was es- 
20 timated. 
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Slowest speed (least angular movement) 



flw ,2jrxl00ipm , 
60 sec 

U 

x \ = £-a) moljr xGcar Ratioxtl = 1°°^ 10,5 sec 1 x-^x-^x 1/24 sec = O024jnm 
x2 = ^a> molor x Gear Ratioxtl = 12^EE.20,9 sec* x ~ x ~ x 1/12 sec = 0 048 ^ 

Scanning time = = 2 * x l - = -^-30 x 30 - 539 seconds « 9 minutes 

<*V« Gear Ratio 10,5 



5 

Fastest speed (maximal angular movement) 

2*rx3000rpm _ _ 
60sec =314 ' 2S6C 

^1 = ^a, wowr x Gear Jtafe x tl = 100 ^ 3 1 4,2 sec 1 x -L x -L x 1 / 24 sec = 0.73 mm 

x2 = ^co motor x Gear Ratio xt2 = 100 ™ 3 14,2 sec" 1 x ^ x ^ x 1/12 sec = 1.45 mm 

Scanning time= = 2n x = -^—30 x 30 = 1 8 seconds 

»M» ^ Gear Ratio 314,2 

If the surface scanned is a continuous varying surface the induced error during the 
10 shutter time may not be a problem, but is the surface consist of sudden changes the 
accuracy will obviously be limited by the angular movement. 

Scanning smaller objects (diameter less than 100 mm) the movement during shutter 
and image transfer will obviously be smaller, and more accurate results might be 
15 obtainable. Furthermore, when capturing an image, the corresponding position is 
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recorded as the encoder position in the middle of the shutter time interval, thus bi- 
secting the possible error. 

The rotation plate 

To minimise errors induced by dynamic mechanics, the rotation plate mounting 
5 system must be stable and accurate. The surface of the rotation plane must be to- 
tally tint, and the axis of rotation must be perpendicular to the plane. Furthermore 
the mounting system on which the plate is fixed must be well designed to obtain a 
high mechanical precision. To enable precise calibration of the system, fixtures were 
desired and milled directly in the plate surface. Although the four fixtures are accu- 
10 rately pos.tioned relative to each other (+/- 5um), the absolute accuracy is not ex- 
pected to be better than +/- 10 um (0.01 mm), due to the "manual" origin calibration 
(using a ^-watch) 

In the process of finding the best solution, for the rotating plate and the mounting 
15 system, two different approaches were implemented. In the first solution tested, the 
plate was mounted directly on the gear. The stability and mechanical accuracy was 
measured using a micron watch. 

The relative horizontal and vertical movement during one rotation was measured 50 
20 mm from the centre axis. Furthermore the vertical displacement caused by adding 
an external force was measured. The result (as shown in table 2) was not satisfac- 
tory and it was decided to create a more stable solution. To control and minimise the 
effect of horizontal and vertical forces the rotating plate was mounted using two 
bearings. Figure 1 1 shows a CAD model of the rotating plate mounting system con- 
25 sisting of a rotating plate 1101, large axial bearing 1102, dust protector and axial 
bearing holder 1103, bottom plate 1104, conventional bearing 1105, bolt fixture 
1106 and gear connector 1 1 07. The bearings used are high precision SKF bearings 
(bearing ball roundness = 0.4 urn). 

30 The stability and accuracy of the new construction was measured, using the same 
technique as for the "direct gear mounting" approach (shown in table 2). As shown 
in the table, the new construction is very stable and indifferent to external forces, 
compared to the first approach. 
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Construction 


Maximal vertical 
displacement dur- 
ing one rotation. 


Maximal horizontal 
displacement during 
one rotation. 


Relative vertical 
displacement 
caused by an exter- 
nal force of 1kg 
=10N. 


Direct gear mount- 
ing 


190|im 


240 jim 


75nm 


Customized design 


8|im 


4^m 


<1^tm 



Table 2: Direct gear mounting versus customised design. Measurement 
of stability and accuracy was performed using a micron watch. 



Steel deformation 

5 The new plate mounting system was tested at various speeds for approximately V* 
hour. Then the construction was disassembled to investigate possible damages and 
changes. The axial bearing and its contact faces were investigated. The axial bear- 
ing had created a visible path, about 0.02 mm in depth 0.2 mm wide. The path was 
constant (of equal size all around) and therefore not considered as a serious prob- 

10 lem degrading the tolerance or accuracy. The path is caused by the softness of the 
steel material used for the bottom plate. The contact surface between the plate and 
the baring balls are infinitely small and therefore the theoretic pressure goes to infin- 
ity. The possibility of using a hardened steel bottom plate was considered. The idea 
of hardened steel was rejected because it could cause the bearing balls to be de- 

15 formed. In that case it is preferable to have soft steel surrounding the bearing balls. 

Clutch 

A play free clutch was installed between the rotating plate and the gear axis. The 
coupling must be flexible though it must still be precise and able to transmit the force 
20 from the gear to the rotating plate. 

Image processing 

The profile generated by the laser light beamed onto the object is captured by a high 
resolution CCD camera, and the resulting image is processed and analysed in order 
25 to locate the lightened contour. This chapter describes a number of different meth- 
ods and techniques to locate points on the laser-generated contour. The majority of 
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described methods features sub pixel detection; thereby it is possible to enhance 
the accuracy of scanned data beyond the resolution of the CCD. Conventional 
methods are investigated, and a fast accurate and stable method is presented. 

5 Image convention 

Most of the presented images are captured using the actual scanner cameras, but in 
order to enhance the visual impression the aperture has been changed to make the 
object visible. Furthermore some contour images have been negated to economize 
with the printer toner. 

10 

The reflected laser contour 

The shape and "quality" of the laser contour imaged by the camera is highly de- 
pendent of the surface of the object scanned. Ideally the surface is planar and dif- 
fuse reflective causing the intensity level along the contour to be constant and the 

15 intensity across to be Gaussian distributed, corresponding to the laser specification 
sheet. Although some objects might satisfy the ideal criteria, it is far from assumable 
that all objects do. If the scanned object is made of blank steel, the reflection pattern 
is much different from the pattern reflected from black plastic. Another factor causing 
large intensity variations in the imaged laser contour is the angle of the surface rela- 

20 tive to the laser beam direction and the camera. If the laser-beam direction is per- 
pendicular to the surface exposed, and the surface is assumed to be an ideal lam- 
bertian surface, the imaged contour intensity is relatively high, corresponding to the 
Bi-directional Reflectance Distribution Function (BRDF) for the ideal lambertian sur- 
face, whereas the reflected light intensity from a surface parallel (or close to parallel) 

25 to the laser direction will vanish. On an ideal lambertian surface, the surface ap- 
pears equally bright from all viewing direction. Its brightness is entirely dependent on 
the angle between the light source and the surface normal. BRDF is used to de- 
scribe the ratio between radiance and irradiance from a surface patch, as a function 
of light source direction and observer position, relative to the surface patch.Three 

30 different reflection types are used to describe the scatter pattern from a given sur- 
face. Figure 12 shows a graphic interpretation of the three different reflection phe- 
nomena, specular, diffuse specular and uniform diffuse reflections from the object 
surface. 
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If the object surface scatter-pattern equals a perfect mirror (only specular reflection), 
the laser light is reflected in only one direction, and the object is obviously not suit- 
able for scanning. For the scanning purpose the ideal surface is the lambertian sur- 
face. Thus the reflected light is uniform diffuse (see Figure 12), causing the laser 
5 light to be spread equally in all directions. Thereby discontinuous light emission and 
ghost points (e.g. caused by secular laser light reflections in the surface) are 
avoided. Later in the overall performance test, the reflection pattern from different 
object surfaces will be discussed. 



10 In addition to the changing light conditions, variations in object shape influences the 
choice and development of a stable contour detection method. Some objects consist 
entirely of planar surfaces (e.g. milled cube), whereas others consist of continuous 
varying surfaces (e.g. clay modelled figure). 



1 5 Based on the analysts of scatter patterns from different objects with various shapes, 
the optimal contour detector must be indifferent to the following artefacts and phe- 
nomena. 

External light from various natural light sources (e.g. lamp or sun-light). 
Changing contour width, and scatter pattern, (e.g. contour on black, white, or 
metal surface) 

Ghost points from reflected laser light. 



20 



It is practically impossible to handle severe artifacts from highly reflective objects 
25 (e.g. mirror). Coating the object surface (e.g. with spray paint) might solve the prob- 
lem, however under some conditions it may ruin the object appearance. Further- 
more the coating will add a thin layer of material to the surface causing the accuracy 
to decline. Under some conditions it may be advantageous to coat the surface and 
thereby obtain a better detection of the laser contour. The thickness of the paint can 
30 be subtracted in the computer after the computer model has been made. The thick- 
ness of the layer of paint can be measured by scanning a known object with a lam- 
bertian surface with and without paint. The paint used to spray the surface may be 
commercial products like Magnaflux ®, Spotocheck ®, or SKD-S2. It may also be 
possible to obtain the desired surface by milling or polishing the surface. 
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Considered contour detectors 

A number of edge and contour detecting methods were considered and tested in the 
search for a suitable solution. 

5 • Edge detectors based on derivatives. 

• Transformation (Hough and radon). 

• Line-by-line search (peak detection, interval midpoint, moment based meth- 
ods and line filtering). 

10 The edge detectors (based on derivatives) and the Hough transform, do not assume 
prior knowledge of the contour direction, whereas the line-by-line methods imple- 
mented assume knowledge of the laser contour direction. 



Edge detectors 

15 The edge detectors that were reviewed and tested are based on zero crossing con- 
tours of first and second derivatives in the different image directions (x and y). The 
tested methods are: 

• First derivative 
20 • Second derivative 

Using derivatives, two edges are detected, one on each side of the contour and thus 
the centre must be interpolated. The two methods were tested in the early phase of 
the project using Matlab, and the tracked contour(s) were unusable. Furthermore the 
25 methods are sensitive to noise resulting in multiple contour detection, was observed. 
Prior threshold of the image to remove noise and dark was tried, but the resulting 
zero crossing contours was still not satisfactory. In addition the methods were found 
to be relatively time consuming. 

30 Hough transformation 

Contour and line detection performed using a Hough transform was investigated and 
tested on a number of different laser-contour images. The idea behind the Hough 
transform Is to enable a simple identification of known image features. In this case 
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the Hough transform was used to identify lines in the image, using the following pa- 
rameterisation to connect image space (x, y) with the Hough space (0 t r): 
x CosO^y Sin0-r. Figure 13 shows a graphic representation of the relation 
between lines 1301 in the Hough and image space. 

5 

To find lines in an image (image space) all points must be transformed into Hough 
space using the relation above. Co-ordinates constituting to the same line equation 
in image space will sum-up the same co-ordinate in Hough space, enabling the line 
to be found using a simple peak detector. The Hough transform is visualised in Fig- 
10 ure 14 and Figure 15, showing the transform applied to the selected area 1401. On 
the left of Figure 14 is shown a selected area and on the right, negative images of 
the contour. To visualise the force and the drawbacks of Hough transform, it was 
applied to the three contours 1402, 1403, and 1404 in the right side of Figure 14. 
The results of the Hough transformation are shown in Figure 15. 

15 

Line-by-line search 

As the camera position relative to the laser is fixed, the laser-beam angle is known, 
and the contour direction can be estimated. If the laser contour direction (search 
direction) is known, the contour position can be detected from the cross section pro- 
20 file (search line profile). This is a simple and easy way to detect the laser contour. 

The search step distance and the profile step (dr) can be varied to optimise for 
speed and accuracy, although it should be noticed that the image pixel intensities 
are discrete values. Pixel intensities corresponding to floating point image co- 
25 ordinates can be estimated using an interpolation scheme (e.g. bilinear or bicubic 
interpolation). In the implemented solutions bilinear interpolation was used to esti- 
mate pixel intensities, corresponding to floating point image co-ordinates. 

The search direction should ideally be changed according to the laser-beam fan 
30 angle, to obtain the most correct laser cross-section profile, and to avoid multiple 
peak detection. However, it is possible to find a search direction that does not cause 
multiple peak detection. It is noticed that the direction of the various bending contour 
cannot be constantly equal to the search direction, using this type of search. 
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The following methods where considered to detect the peak of the search line pro- 
file: 



Max intensity 
Centre of gravity 
Filter search 



Max intensity 

The simplest way of detecting the profile peak is to find the global maximum. How- 
10 ever it is not possible to gain sub-pixel accuracy, in the discrete image. 

Centre of gravity 

A more accurate way to locate the peak, is to calculate the centre of gravity for the 
profile. The estimated peak position can be calculated as: 



15 



Although it is possible to gain sub pixel accuracy using this method, it is highly sen- 
sitive to image noise, and dark current in the CCD-chip. If the peak is not located in 
20 the middle of the watched interval, the image noise sum up unsymmetrical and the 
peak detection might be pulled aside. To remove noise and dark current, a prior 
threshold of the image can be performed, thus introducing a new problem: finding 
the right threshold level. The threshold level must be high enough to remove the 
noise, and still be small enough to enable sub pixel accuracy. 

25 

Filter search 

Another method that can be used to locate the profile peak is filter search. By multi- 
plying a pre-constructed window filter with the image along the search line, the peak 
is found where the maximum response is present. The method is somewhat insen- 
30 sitive to noise in the image, and the preliminary threshold is thereby avoided. As the 
laser contour cross-section intensity profile is assumed Gaussian distributed, it is 
logical to choose a Gauss filter to search for the peak centre. 
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In some cases it might be preferable to have a circular window filter, but to ease the 
implementation all filters were implemented as rectangular window filters. 



Calibration and transformation 

5 

In the following a number of different calibration techniques are discussed and dis- 
closed. A calibration object according to the invention being designed exclusively for 
the calibration purpose is disclosed. The calibration leads to the direct transforma- 
tion of co-ordinates between the defined co-ordinate systems. A method for direct 
10 calibration according to the invention is thereby also disclosed. 

To obtain a 3D point cloud representing the scanned object, the contour points de- 
tected in 2D (the image co-ordinates) must be transformed into real world 3D co- 
ordinates, and rotated according to the plate rotational angle. To simplify the trans- 
15 formation and enable use of well-known methods to calibrate the camera and lens 
(e.g. DLT, Heikkila or Tsai), the transformation is performed in two steps, namely: 

• Image co-ordinates (2D) -> Laser sheet co-ordinates 

• Laser sheet co-ordinates World co-ordinates (3D) 

20 

To assure correct transformation of points between the different co-ordinate sys- 
tems, the involved transformations must be calibrated prior to the actual scanning. In 
this case the calibration is performed beaming the laser line onto a known physical 
object, thus obtaining knowledge on the relationship between the different co- 
25 ordinate systems. 



Co-ordinate systems 

As shown in Figure 16 three co-ordinate systems are introduced to describe the 
transformation from image co-ordinates to 3D world co-ordinates. Those are the 
30 image co-ordinates system (X, ,Y t ) 1601, the laser sheet co-ordinates system (Xl 
,Y l ) 1602 and the 3D real world co-ordinates system 1603 (X, Y, Z), also referred to 
as the main co-ordinate system or object co-ordinate system. Note that there is a 
one to one mapping 1604 between 3D coordinates 1605 in the laser plane 1606 and 
2D image coordinates 1607 through a pinhole (lens) 1608. As said earlier the 
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transformation is performed in two steps; from image co-ordinates to laser sheet co- 
ordinates and then from laser sheet co-ordinates to real world co-ordinates. In the 
following sections the two transformation steps are treated individually, although 
they are based on the same basis geometric assumptions, of which the most im- 
5 portant are: 



• The Y-axis of the main co-ordinate system (O) is assumed equal to the axis 
of rotation for the plate, and thus the Y-axis vector equals the plate surface 
normal. 

10 • The plate surface is assumed totally flat. 

• The laser sheet is assumed totally flat. 

• The CCD array (image plane) is assumed totally flat. 

• To assure a one to one and onto (One to one and onto: A co-ordinate in im- 
age space corresponds to one and only one co-ordinate in the laser sheet 

15 plane, and vice versa) transformation between image and laser sheet co- 

ordinates, the image plane normal cannot be perpendicular to the laser sheet 
normal (i.e. the camera must be angled relative to the laser). 

• To assure a one to one and onto transformation between laser sheet and 
world co-ordinates, the laser sheet normal must be somewhat perpendicular 

20 to the (Z, Y) plane. 

Image plane ^> Laser sheet 

This section deals with the transformation of co-ordinates from image plane to the 
laser sheet, and the prior calibration needed for the transform to be correct and ac- 
curate. Four methods have been considered: 



25 



30 



• Direct linear transformation (DLT). 

• Tsai algorithm [1] proposed by Rodger Y. Tsai. 

• Heikkila algorithm [3] proposed by Janne Heikkila. 

• Direct non-linear matrix transformation. 

As light rays travels from the laser sheet (intersection with the scanned object) to the 
image plane (CCD) they may well be distorted in the camera lens system, or hit a 
wrong CCD pixel element due to inhomogeneous pixel size and pixel spacing on the 
CCD. Neglecting these distortion effects, the transformation of co-ordinates between 
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the image plane and the laser sheet can be modelled using the so-called pinhole 
model. The pinhole model describes the central perspective projection (as shown in 
Figure 16), of coordinates in space onto the image plane. The pinhole model as- 
sumes all light to pass trough an infinitely small hole 1608, which is obviously not 
5 practically possible. To avoid long exposure time due to the limited pinhole size, a 
lens is inserted at the pinhole position thus redirecting all light-rays from the same 
point in space to hit the same co-ordinate on the image plane. Although the ideal 
lens will redirect all light rays towards the right CCD pixel element, it is not assum- 
able that a standard off-the-shelf lens will be able to do so. For this reason the lens 
10 distortion must be handled in order to obtain a precise model for the transformation. 

Direct linear transform 

Neglecting non-linear lens distortion the transformation of co-ordinates from laser 
plane to image plane can be formulated as a direct linear transform (DLT). 
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Where w is an arbitrary scaling factor allowing the image perspective to be mod- 
elled. a u are constants that must be calibrated (determined) prior to the actual scan- 
ning. Restricting one of the constants (a^l), the existing constants in the linear 

20 equation system can be calibrated (determined) from a minimum of 4 corresponding 
point sets (see Figure 16). Using more than four points to calibrate the transforma- 
tion constants a least squares approach is used. A comprehensive description of the 
mathematics constituting to the equation set-up can be found elsewhere, e.g. a 
standard computer vision text book or www.icaen.uiowa.edu/-dip/ LEC- 

25 TURE/3DVisionP1 2.html . Results and accuracy of the transformation are pre- 
sented and discussed below. 

Tsai 

To obtain a more accurate transformation of co-ordinates, Rodger Y. Tsai has for- 
30 mulated a calibration method [1] often referred to as the two-stage Tsai calibration 
technique, allowing first order radial lens distortion to be modelled[1]. Furthermore 
the technique includes a calibration method optimised for coplanar observations, 
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which is specially suited for this application (the laser sheet is assumed coplanar). 
Although the Tsai calibration method and transformation is used, the mathematics 
behind it will not be described in application. The method has been described in a 
number of publications and can be found elsewhere [1]. The implementation by Reg 
5 Wilson (TSai calibration, implementation by Reg Wilson: www.mmm.com), found on 
the web was adopted and a DDL (Dynamic Link Liberay, used to interface between 
different windows based programming platforms (e.g. C++ and Delphi), interface 
was created to enable Delphi to call the functions and procedures written in C. Re- 
sults and accuracy estimates are presented and discussed below. 

10 

Heikkila 

An alternative calibration method along the same line as the Tsai is proposed by 
Heikkila [3]. Compared to the Tsai algorithm the Heikkila calibration method consists 
of a four-step procedure, where all the calibration parameters are simultaneous op- 
15 timised using a bundle adjustment approach including a non-linear optimisation al- 
gorithm. For details about the algorithm refer to [3]. 

Direct non-linear matrix transform 

Assuming the transformation from image co-ordinates to laser sheet co-ordinates to 
be one to one and onto (One to one and onto: A co-ordinate in image space corre- 
20 sponds to one and only one co-ordinate in the laser sheet plane, and vice versa), 
the following relationship can be formulated: 

X L = f(X it Y,) , for one and only one combination of X ; and Y s 
Y L = g{X { , Y ( ) , for one and only one combination of X, and Y f 

25 Instead of using a simplified model of the real world (as modelled by the DLT, Heik- 
kila and Tsai algorithm) to solve the transformation, one can "expand" the transfor- 
mation functions (/ and g) to be "infinitely" accurate, using a matrix transformation 
(lookup table) for each function. Obviously the transformation matrix must be filled 
with values that are more accurate than what can be calculated using DLT, Heikkila 

30 or Tsai (the idea is not just to create lookup tables reflecting simplified models of the 
real world). Consequently it is required that function values of f and g must be found 
for all image co-ordinates, resulting in two infinitely large matrices (2D). In practice 
this is not possible, but it should be possible to get sufficiently close using our prior 
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knowledge about the system, and limiting our demand for accuracy. First of all the 
transformation is assumed to be continuously defined, for all image co-ordinates (i.e. 
Sudden changes in the lens surface or the CCD array is not expected). As the func- 
tion is continuous, adequately accurate results can be obtained using interpolation 
5 between neighbouring matrix cells (e.g. bilinear or bi-cubic interpolation), thus limit- 
ing the size of the matrix (lookup table) describing each transformation f and g. Fur- 
thermore the matrix can be limited to include only co-ordinates on the laser sheet 
that are visible to the camera. Using the CCD rows and columns to limit the resolu- 
tion of the array and bilinear interpolation to determine the function values in non- 
10 discrete image co-ordinates; the transformation functions can be expressed as fol- 
lows: 



X L = f(X n Y,) = {\-dx).(l-dy).f array [xr e ,Yr e ] 

+ dx-(i- dy) ■ f orray \xt + 1, rr c ] 

+ dy(i-dx). / im Jxr ae ,Yr e + 1] 

+ dx-dyf amy [xr° +U1J—+1] 



15 



20 



Y L =g(X ( ,Y,) 



= (i-dx)(i-dy) ■g arny [xr c ,Yr t J 
+dx(i-dy). garray [xr c + 1, rr* \ 
+ d y (i-dx). gamv [x'r , Yr e + 1 

+ dx-dyg array [xr ne +l,Yr c +l] 



Where Xf" ne and Y*" ne are the integer parts of the X, and Y,. dx = X t -Xf u " e and 
dy = Y t — Yf™* . The size of f^y and g array matrices equals the camera CCD size 
(1300 x 1025 pixels), and thus the memory needed to store the each lookup table 
(matrix) is: 1300 • 1025 • 4 « 5Mb (4 byte float = 7-8 significant digits). 



Although a suitable transformation have been defined, one has to fill in correct and 
accurate values in every cell of the transformation matrix (lookup table), to make it 
work. Theoretically 1300x1025*1.4 million correct placed and corresponding co- 
25 ordinate sets connecting the two systems, is needed to fill the matrices. However it 
may well be possible to reach a sufficiently accurate result using a reduced number 
of observations, thus using interpolation to fill in the "holes", based on the continuity 
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assumption. To obtain the actual co-ordinate sets, it one may use a translating cali- 
bration object as shown in Figure 17, thereby recording multiple corresponding co- 
ordinate sets extracted from the known corner features 1701. The object in Figure 
17 is used to calibrate a system to a matrix transform. 

5 Laser sheet <-> real world 

To complete the overall transformation of image co-ordinates to 3D world co- 
ordinates, the relationship between co-ordinates in the laser sheet (X L ,Y L ) and the 
mam coordinate system (X,Y,Z) must be defined. As the object is rotating it in- 
cludes a rotation around the Y-axis. Figure 18 illustrates the relationship between 
10 the two coordinate systems. In Figure 18 (a) is shown the definition of q>=0 (encoder 
position = 0) In Figure 1 8 (b) is shown the rotation relative to the zero angle. 

The transformation from laser sheet co-ordinates (X L , Y u ) to real world co-ordinates 
(X.Y.Z) can thus be defined by the following equation: 

15 



"a" 




COS(£») 


0 


sin(^) 


~K-Y L -d 


Y 




0 


1 


0 




Z 




-s\n(<p) 


0 


cos(#>) 





where = tan(cr), k 2 = cos(a). 

Ideally the laser sheet equals the (Z, Y)-plane, consequently the transformation can 
20 be expressed as a simple counter clockwise rotation around the Y-axis (i.e. a=0 and 
d=0 => /c,=0, /c 2 =1 ). However it is practically impossible to adjust the laser sheet to 
be that accurate, even using the fine adjustment mechanics, therefore d and a must 
be calibrated (found) or at least estimated prior to the transformation. In addition the 
transformation can be used in a multiple laser setup, with several laser sheets. 

25 

Choice of calibration object 

To calibrate the adopted transformations, image features corresponding to known 
laser sheet co-ordinates must be recorded. 
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In addition the laser sheet parameters (d and a) must be calibrated based on the 
same feature detection or additional calibration. To enhance calibration performance 
image features^must be detected within sub pixel accuracy (the resolution of the 
used CCD is relatively coarse, approximately 0.1mm pr. pixel using a 25mm lens at 
400-500mm distance). Corresponding co-ordinates of different systems can be ob- 
tained using a calibration object placed on the rotating table. The following consti- 
tutes to the design and choice of calibration object: 

• Ability to calibrate the whole system, both transform from image co-ordinates 
to laser sheet co-ordinates, and the transform from laser sheet co-ordinates 
to object co-ordinates. 

• Fea tures d etected in image space must be known in world co-ordinates. 

• Feature should be easy to detect using the desired lens aperture stop. 

• The fe ature d etection should be stable and versatile. 

• The designed object must be feasible, even at high accuracy. 

• The calibration object should have as many accurately detectable and known 
features as possible, spread out on the area of interest 



Based on the stated demands, a number of different calibration objects may be con- 
sidered; the most important ones are shown in Figure 19. Figure 19, 1901 shows a 
prior art calibration object known as the traditional dot matrix. 1 902 shows a calibra- 
tion object in which the laser line is used to generate the features thus enabling cali- 
bration of both laser, camera and lens system. In 1903 is shown a preferred em- 
bodiment of the inventive calibration object comprising two symmetric and continu- 
ous winding staircases. 

To calibrate a camera-lens system the dot matrix 1901 is commonly used. The dot 
matrix is available as an off-the-shelf component from a number of vendors (e.g. 
Melles Griot), and the accuracy is high (typ. 5 fim). Furthermore, the dots are easy 
and accurately detected using a simple blob detection algorithm. Despite these ad- 
vantages of the dot matrix, it does not allow the laser sheet to be calibrated simulta- 
neously. Consequently the laser sheet position relative to the world co-ordinate 
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system must be calibrated subsequently (e.g. using another object). Furthermore it 
might be necessary to determine the axis of rotation thus introducing a number of 
error sources. Another crucial problem is the fact that the lens-aperture stop is likely 
to be changed subsequent to the calibration to compensate the laser light intensity. 

Exploiting the geometry of the rotational plate and linear axis combined with the fact 
that all essential co-ordinates are located in the same plane (the laser sheet), four 
alternative calibration objects according to the invention were designed (i.e. 1902, 
1903, 1904 and 1905). 1903, 1904 and 1905 is a preferable embodiment of the cali- 
bration object since it has the following advantages (compared to 1902): 

• Continuous geometry enabling, sub-sub pixel accuracy. 

• Limited occlusion (continuous geometry compared to the stairs) 

• The symmetric geometry enables the laser sheet parameters to be calibrated 
using the same object <p = 0,d, a. (see Figure 18 and Figure 28). 

• Feasible and somewhat easy to produce object 

• Surface features (3D object feature curves) defined by known mathematical 
functions facilitate the calculation of co-ordinates for calibration. 



Calibration object geometry 

The calibration object 1903 was designed to provide known co-ordinate features as 
a function of the rotational angle throughout the desired scan area. Although it is 
preferable to have feature points throughout the scan area, it is possible to calibrate 
with the same object using a larger FOV (field of view). Utilising the geometry of a 
winding staircase (continuous), the features detected at the intersection between 
object and laser sheet will follow continuous known curves (assuming d«o»0, the 
features are lines). Thus enabling a simple and accurate feature extraction process. 
Furthermore the object is symmetrically designed to allow the laser sheet parame- 
ters to be calibrated simultaneously. Figure 20 shows how feature points 2001 can 
be extracted from a captured image given the invented calibration object 2002 and 
the laser sheet 2003. Figure 22 shows the corresponding image where 3D object 
feature curves 2201, 2D image feature curves 2202 and image feature points 2203 
are visible. 
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As illustrated in Figure 20 the feature co-ordinates in image space can be calculated 
from the intersection of two lines. The corresponding laser sheet co-ordinates are 
calculated from the rotational angle, <p, provided by the encoder. 

> The calibration object was modelled in a CAD system the object by connecting a 
number of mathematically defined spirals, forming a solid body. The eight 3D object 
feature curves 2101 were created from the definitions below and are plotted in Fig- 
ure 2 1 

U Id 
r| ((p) = 59-10m , y . i ( < p ) = y off + 8om 

71 7t 

r 2 (<p) = -52.5 + 10-lol^ , y 2 (<p) = y OI r + 80-8oM 
r 3 (<p) = 48-l()M , y' 3 (q>) = y OI r + 8oM 

71 It 



r 4 (<p) = -41.5 + 10-loM , y' 4 (<p) = y ofT + 80-80^ 
7i n 



+ 80-80^1 



15 r 5 (cp) = 37-loM , y' 5 (cp) = y 01T + 8oM 

io 7i n 

M 

r o (<p) = -30.5 + 10-10^ , y , 6 (cp)=y off 

7t 

r 7 (cp) = 26 + loM , y 7 (cp) = yofr + 8oM 

71 71 

r t (tp) = -Not used - 
20 forcpe[-7i;7t] 

As will be immediately evident for the skilled person, numerous variations of the 
mathematical functions describing the 3D object feature curves can be constructed, 
keeping the basic requirements of the object in mind: continuous and symmetric 
25 geometry, limited occlusion, coverage of the whole camera field of view. 

Another advantage of the present calibration object is that it is possible to construct 
the object in a CAD and make it using automatic milling controlled by the CAD. 
Thereby great precision of the individual 3D object feature curves is obtained. 

30 

Assuming d=0 and a=0 (see Figure 18), the laser sheet co-ordinates corresponding 
to detected feature points can be found directly from the rotational angle using the 
3D object feature curve definitions: ^. 

35 ='}(*>) , Y L ^y\{<p) 
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However it is practically impossible to obtain d=0 and o=0, consequently a small 
angular displacement is included, resulting in the following update of the equations: 

5 X LJ = r t <jp + d<p(d,a,<p)) , Y UI = y\(<p + d<p(d,a,<p)) 

Assuming the d«aw0 (very small) the angular displacement d<p can be approxi- 
mated. 

10 \axi(d(p) * — 

Shiny surface 

The initial result of the milling, was a "steel glossy" calibration object with a mirror 
like surface. To improve object reflectance, the calibration object was treated with an 

15 etching liquid (NaOH). The resulting object surface (see Figure 22) was less specu- 
lar and almost optimally diffuse reflective. As a result of the treatment the feature 
detection stability and accuracy improved significantly, however it did also remove 
approximately 0.04 mm of the object surface and thus the feature parameter de- 
scription had to be adjusted. The object shown in Figure 22 was milled using a high 

20 speed CNC milling machine. The milling time was approximately 1 hour with an ab- 
solute accuracy better than +/- 0.02 mm. Such high precision is required for accu- 
rate calibration precision. 

Another way of dealing with this problem is to spray the surface with paint, charcoal 
25 dust, chalk powder or other similar materials that cover the surface and provide the 
surface with the desired lambertian characteristics. Whereas it is important for all 
surfaces to be scanned to have the desired surface characteristics it is especially 
important for the calibration object, since errors in the calibration affect all subse- 
quent measurements done by the scanner. 

30 Feature detection 

With the accomplished design of calibration object, the feature extraction is reduced 
to line and corner detection. Contour points can be found using the Hough or Radon 
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transform. The corners are found as the points where the contour points change 
direction, in Figure 23 is illustrated how contour points 2301, and corners 2302 are 
detected with sub-pixel accuracy. 

5 

Multiple features 

Figure 24 shows the corner features detected during a full rotation of the calibration 
object (suppose that the object in Figure 22 rotates, while corner features are de- 
tected and recorded). Figure 24 shows a screen dump from the calibration point 

10 processor program. In Figure 24 (a) are shown raw corner features 2401 detected 
during one full rotation of the calibration object (seen from camera 1). In Figure 24 
(b) are shown manually selected 2D image feature curves, which correspond to the 
calibration object 3D object feature curves. In the shown example the rotation was 
performed in one minute, resulting in approximately 1300 valid feature points (6 

15 frames per second, 7 valid points per frame) with corresponding rotational angles 
(encoder position). 

Although corners are detected using sub pixel accuracy, there are still a number of 
outliers and undefined comers that does not correspond to the parametric feature 

20 definition in Figure 21 (especially at angles close to 0 or +/-*). These corner features 
must be deleted, before the actual calibration can take place. Furthermore the cor- 
respondence between "corner lines" (see lines in Figure 24) and the parametrically 
described 3D object feature curves (see Figure 21) must be determined. In the im- 
plemented setup the reduction and identification was performed using a semi- 

25 automatic user interface, requiring the user to point out the approximate position of 
the different 2D image feature curves (i.e. using a mouse). 

Displaying the detected comer features x-co-ordinate (image space) as a function of 
the encoder position reveals the result shown in Figure 25. The symmetry points 
30 2501 will be used in the next step to find the encoder position corresponding to q>=0 
(see Figure 21), and calibration of the defined laser sheet parameters (see Figure 
18). 

During the first half of one rotation (-tc to 0), the calibration object is facing against 
35 the camera, and thus corners are easily and accurately detected. At the second half 
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of one rotation (0 to 71) some features may not be detectable due to occlusion, and 
faulty detection might occur (front and back facing see Figure 26). However some 
corners detected during the second half (0 to %) are necessary to calibrate the laser 
sheet (exploitation of symmetry). When the laser sheet parameters have been de- 
5 termined, the camera-lens calibration can be completed (DLT, Heikkila or Tsai), us- 
ing only the corners detected during the first half round (-n to 0), thereby avoiding 
that inaccurately detected feature points spoils the calibration result. 

Laser sheet calibration 

10 The laser sheet parameters can be calibrated by analysing the encoder position 
corresponding to coexisting (symmetric) image features 2501 as shown in Figure 25 
and 26. Recording a number of likewise symmetrical coordinates, enables the cali- 
bration of the laser sheet position (d and a), relative to the coordinate system (see 
Figure 18). Notice that the object features may be occluded by the object itself. 



The laser sheet may be parameterised by the following parameters (see Figure 18): 

• Encoder offset, <p 0 , corresponding to ^0, locking the co-ordinate system. 

• Laser sheet displacement, d. 

• Laser sheet angle, a. 



To obtain a good estimate of the encoder offset, tp 0 , corresponding to p=0,and en- 
able estimation of cf, the mean encoder position 2701 was calculated for all symmet- 
rical points (shown in Figure 27). In the Figure encoder offset and laser sheet pa- 
rameters are extracted from encoder mean position of symmetric image features 



15 



20 



25 



(see Figure 25). 



Using the sinus relation, cf can be approximated by: 



sin(v) 



sin(0) 



=> d 



^ r /+2 sin(v) r 
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W here n are defined in figure 28. Note the laser plane 2801 and the rotation centre 
2802. When d is known (approximated), the encoder offset corresponding to <p=Q 
can be approximated: 

_ , ^ 9 • Tics per round 
5 Encoder offset = pos _ r s 

The laser sheet angle a is rouahlv estimated from the slope of the first order ap- 
proximations 2702 (Figure 27), using of as the distance reference. 

10 Using the formulas above the laser sheet parameters were estimated to d«0.1 mm, 
^146210, ct«0.11°. The noise level on each mean value is seen to be relatively 
high (100 pulses corresponds to a relative change in the 2D image feature curve 
position y direction of approximately 0.045 mm). Furthermore it can be seen that the 
noise on the corner features located far from the camera (line 2) is much higher than 

15 the noise on features close to the camera (e.g. line 1). 



Furthermore a first order approximation was calculated to determine the relative 
movement of symmetrical points and thus enabling approximation of the laser sheet 
angle a. 

20 

An alternative method for estimation of the laser sheet parameters based on pairs of 
symmetric points is to create a <p-0 plot, where <p is the rotation angle / translation for 
the first symmetric point and 0 is the difference in the rotation angle / translation 
between the symmetric points. For each of the visible feature curves on the calibra- 
25 tion object a corresponding line will appear in the cp-e plot It can be shown that 
these lines usually correspond to straight lines for a and d close to zero. Likewise 
mathematical formulas can be derived for these lines, where the only unknown pa- 
rameters are a, d and <p 0 . Given two or more lines in the <p-Q plot the laser sheet 
parameters can be estimated using the derived formulas. 

30 
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Laser intensity calibration 

The laser intensities are preferably varied depending on the surface and colour of 
the object to be scanned. Preferably the intensity should be determined automati- 
cally using automatic laser intensity calibration. 

5 

The intensity calibration may be performed by inserting the object into the scanner 
and calculate a number of histograms from the acquired images. First a histogram is 
calculated with the laser turned off. A second histogram is the calculated when the 
laser is turned on with an arbitrary intensity. The first histogram is then subtracted 
10 from the second to remove the background intensity. The intensity is then adjusted 
until the selected quantile corresponds to a predefined intensity. The background 
could also be removed by subtracting the image corresponding to the laser turned 
off from the image with laser light. The histogram used to determine the intensity 
could then be calculated from this difference image. 

15 Calibration performance 

To investigate calibration performance, feature points (X h Y,) are back projected to 
the laser sheet, and compared with their original counterparts (X U Y L ): 

Tsai/DLT(X ; , Y l ) = (X L Tsai , Yj^ Tsai ) compared to (X L ,Y L ) 

20 

Three feature sets (e.g. 685 corresponding image and laser sheet co-ordinates) 
were recorded using the same camera-laser setup. DLT, Heikkila and Tsai calibra- 
tion was performed, and the image co-ordinates were backprojeced to measure the 
model accuracy. The result shown in table 3 was obtained with camera 1, positioned 
25 relative to the laser as shown in Figure 22. Calibrating camera 2 leads to approxi- 
mately the same results, although it is obviously somewhat dependent on the cam- 
era position relative to the laser. 
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Feature 
set nr. 


d 


a 


Method 


Removed 
outliers 


Points 




Mean 
deviation 


Std. 

deviation 


Max 

deviation 


1 


0 


0 


DLT 


0% 


685 




0.63 


0.41 


1.3 


2 


0 


0 


DLT 


0% 


1470 




0.58 


0.38 


1.8 


3 


0 


0 


DLT 


0% 


1295 




0.62 


0.40 


1.8 




1 


0 


0 


Tsai 


0% 


685 




0.0227 


0.0190 


0.12 


1 


0.08 


0.11 


Tsai 


0% 


685 




0.0211 


0.0188 


0.10 


1 


0.08 


0.11 


Tsai 


26% 


503 




0.0072 


0.0056 


0.03 




2 


0 


0 


Tsai 


0% 


1470 




0.0249 


0.0194 


0.16 


2 


0.08 


0.11 


Tsai 


0% 


1470 




0.0235 


0.0190 


0.14 


2 


0.08 


0.11 


Tsai 


33% 


982 




0.0073 


0.0063 


0.04 




3 


0 


0 


Tsai 


0% 


1295 




0.0221 


0.0166 


0.18 


3 


0.08 


0.11 


Tsai 


0% 


1295 




0.0219 


0.0173 


0.13 


3 


0.08 


0.11 


Tsai 


30% 


982 




0.0075 


0.0059 


0.03 



Table 3: Errors on backprojected image features compared with their original 
counterparts (data have been captured using camera 1). Calibration of camera 
2 revealed somewhat the same results. Removing outliers, improves the re- 



5 suit, although it may not correspond to real improvements in terms of more 
accurate transformation parameters. All measurements are in mm. 

As shown in Table 3 the standard deviation using the TSai calibration technique is 
approximately 0.01 mm, corresponding to a standard deviation in image space of 

10 approximately 0.1 pixels (measured on backprojected laser sheet co-ordinates). 
Considering the accuracy of the milled calibration object, the potential mechanical 
error sources and non-modelled lens distortion etc. it is an extremely high accuracy. 
However, the calculated errors are relative and based on the same points as used 
for the actual calibration, thus the measured error cannot be used as a direct quali- 

15 fier for the scanner accuracy. 

Instead the error can be used as an indicator of the calibration model accuracy (Tsai 
versus DLT), and the magnitude of the feature detection noise. As shown in table 3 
the Tsai calibration and transformation technique, models the camera and lens dis- 
20 tortion much better than the direct linear transformation. Consequently the main er- 
ror source using the DLT transform, is assumed to be the DLT transformation model 
itself (i.e. the perspective projection does not model non-linear lens distortion). 

Using a Tsai calibration to model the camera-lens system the error is much smaller. 
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Removing the worst outliers (25-30%) and then re-calibrating, obviously reduces the 
error on back projected points, although it may not correspond to more accurate 
calibration parameters. Small changes of the laser sheet parameters (d and a) does 
5 not influence the back projection result significantly, although it changes the calibra- 
tion parameters and thereby the overall transformation accuracy. 

For each camera in the setup, the TSai/DLT calibration needs to be repeated. It is 
straightforward to add additional cameras, thus reducing occlusion effects and en- 
10 hance scanning accuracy. 

Overall performance 

The overall performance of the scanner system has been tested by a number of 
15 relevant scanner jobs, and has proved to yield result of very high quality, surpassing 
competitive solutions. 

Ear model 

The developed system's, ability to scan ear models was investigated. 

20 

Figure 29 shows Camera images (2D) of an ear impression supplied by a hearing 
aid manufacturer. The model corresponds to the inside of a human ear, and it is 
used to create a fitting hearing aid, customised especially for the patient. Currently 
the hearing aid is created in a silicon mould, made with the ear clay model. 

25 Result of scanning the ear model shown in Figure 30. The shown 3D model consists 
of 332,955 triangles spanned between 166,843 points. The 3D model was created 
from two sub scans that have been registered and merged. Another setup more 
suitable for such scanning is the scanner disclosed in Figure 4,5 and 9. Using one of 
these scanner set-ups it is possible to scan the model in one scan. Improved scan- 

30 ning is especially obtained when the model is placed on a spike as in Figure 6. 

The shown 3D model was scanned and processed in approximately 3 minutes, 
however it is possible to complete the 3D model in less than 1 minutes if the regis- 
tration process is automated, or avoided (e.g. by introduction of one or more cam- 
eras). 
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As shown by the figures it is possible to scan and create very detailed and accurate 
copies of ear clay models with the developed system. 

5 In order to improve the detection of surface features during the scan, it may be ad- 
vantageous to treat the surface of the clay model as described above to obtain a 
lambertian surface. 

As described, hearing aids are normally made by taking an impression of the ear 
10 canal. This is normally done by educated staff. The impression is then sent to the 
hearing aid manufacturer, who casts a mould around the impression and moulds the 
shell inside. Thereby, at least 3 impressions are made, each of which contribute to 
the error of the hearing aid shell. The scanning methods disclosed in this application 
are very useful for comparing the size of the impression to the size of the moulded 
15 or hand made hearing aid shell as a quality control, before the hearing aid shell is 
sent to the patient. The comparison can be made in a computer, where the result of 
a scan of the impression is compared to the result of the scan of the hearing aid 
shell. A colour code can be used to identify the areas where the differences differ 
more than a pre-determined tolerance. 

20 

In many cases, the hearing aid shells are returned by the patient to the hearing aid 
manufacturer due to lack of fit. In these cases it is advantageous for the manufac- 
turer to be able to compare the actual size of the impression to the size of the hand 
made hearing aid shell. 

25 

Devices for the ear 

Other devices that could advantageously be incorporated into a shell manufactured 
according to the disclosed method include mobile phones, communication devices, 
loud speakers, tinnitus masking devices, or devices recording vibrations in the skull 
30 and transforming these into an audio signal. 

Devices that may be incorporated into a shell in the ear also comprise devices re- 
lated to Man Machine Interface (MMI) products, such as custom made ear micro- 
phone/receivers that enable reliable and clear communication even in the noisiest 
35 environments, or products related to wireless internet applications. 
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Speech not only creates sound waves, it also generates vibrations within the skull of 
the speaker. These vibrations can be picked up in the ear (they may be picked up 
other places too, but by far the most convenient method is to do it in the ear). In one 
5 piece, a device thus may comprise a microphone to pick up the speech of the per- 
son wearing it, and a loudspeaker to transmit the speech of the communication 
partner It is important that such devices are made to fit the ear. 

The devices based detection of vibration instead of sound can be used in the noisi- 
10 est environments, since they only pick up the speech of the wearer and they allow 
for quiet communication, since the speaker can speak with a low voice when 
needed. The devices allow for completely hand-free communication. 

Such a device is naturally also devoid of any kind of acoustic feed-back. 

15 

Scanning the ear canal 

Another possibility according to the invention, is to scan the ear canal directly. This 
can be done by building the components of the scanner into an apparatus, which 

20 can be inserted into the ear of the patient. Embodiments of this scanner are shown 
in Figure 31. Preferably, the light source, e.g. the laser, and the camera are located 
outside the ear. The laser light can be carried into the scanner by light guides 3101 
such as optical fibers, and similarly, the reflected signals are carried to a camera by 
image guide 3102 such as optical fibers. During the scan, the scanner preferably 

25 rests on the edge of the ear canal, most preferably in those places where bones are 
closest to the skin surface. This is in order to obtain the highest stability and is very 
important, since the scanner itself works with an accuracy less than 0.05 mm. The 
length of the ear canal can be scanned by moving the scanner in or out and record a 
series of overlapping images of the ear canal. The scanner may comprise only one 

30 laser source and one camera as the one shown in the left of Figure 31 In that case 
the scanner has to rotate while the camera records images. Preferably, the scanner 
comprises multiple laser sources such as four as shown in the scanner in the right 
part of Figure 31. The presence of multiple laser sources and cameras removes the 
need for rotation of the scanner in the ear canal. In the laser scanner disclosed in 
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Figure 31 , the laser source or sources project a ray of laser light on the surface of 
the ear canal. 

Another type of ear canal laser scanner is shown in Figure 32. Here the laser light is 
5 projected as laser sheets producing a laser contour on the surface of the ear canal. 
Thereby, more rapid scanning can be performed compared to the above laser scan- 
ner. In the scanner shown in the right part of Figure 32, four laser sheets 3201 and 
four cameras 3202 are present. Thereby the laser sheets cover the whole circum- 
ference and rotation of the scanner is not required. 

10 

A third type of ear canal scanner is shown in Figure 33. The scanner is shown in the 
ear canal 3301 and consists of a rigid probe/pipe 3302, a box 3303, light guides 
3304 such as optical fibers, a light source 3305 such as a laser, a position sensor 
device 3306, a camera 3307, optics 3308 and a collision detector / distance meas- 

15 ure 3309. This scanner works by projection a structured light pattern such as a grid 
of spots on the whole circumference. The light is generated by the light source and 
travels through the light guides. At the end of the light guides optics and mirrors cre- 
ates the desired pattern. The light beams 3310 for a dot pattern are illustrated in 
Figure 33. By locating this pattern the corresponding 3D surface positions can be 

20 reconstructed adopting the techniques described above. The rigid canal scanner 
has the advantage that light guides such as optical fibers are not required for the 
camera. Note that light guides degrade the image quality. Another advantage of the 
rigid scanner is that the position sensor can be placed outside the ear, e.g. in the 
box, and not on the head of the probe/pipe. Examples of position sensor are mag- 

25 netic sensors, optical sensors, sonic sensors and mechanic sensors. An alternative 
to measuring the positions is to register each local scan of the subsurface to form 
the full surface. This technique is feasible since only the relative positions of the 
subscans are required to form the full surface. 

30 The disadvantage of the rigid scanner is the lack of flexibility to move inside the ear 
canal, which can be obtained by a flexible probe/pipe. The embodiment of a flexible 
scanner in the ear canal 3301 is shown in Figure 34. The scanner consists of a 
flexible probe/pipe 3302, a box 3303, light guides 3304, image guides 3401 such as 
optical fibers, a light source 3305 such as a laser, a position sensor device 3306, a 

35 camera 3307, optics 3308 and a collision detector 3309. 
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A design that improves the image quality of the flexible ear scanner is shown in Fig- 
ure 35. The difference is that the camera 3307 is moved to the head of the 
probe/pipe to avoid degrading of the image quality through the light guides. In a 
5 similar way the camera can be moved to the front of the probe/pipe for the rigid 
scanner. 

The same types of variation of the ear canal scanner can be used as in other cases 
of three-dimensional scanners according to this invention. Thus, preferably the 
10 scanner comprises at least two cameras, more preferably 4 cameras such as for 
example 6 cameras. Likewise, there may be several laser sources such as for ex- 
ample 2 lasers creating laser sheets with an offset of 180°, preferably 3 laser sheets 
with an offset of 120°, or 4 laser sheets with an offset of 90°. 

15 Prior to scanning, the patient's ear must be rinsed to remove cerumen. In some 
cases it may also be advantageous to treat the surface of the ear canal if the ap- 
pearance is too glossy. This can be done in the way described above for obtaining a 
lambertian surface. 

20 One scan can be performed in less than 1 minute, and it is thus possible to record a 
number of scans of one ear, and ask the patient to deliberately vary the size of the 
ear canal by swallowing, yawning, chewing, and drinking. In this way a series of 
scans of the ear canal can be recorded and the magnitude of the variation of the 
patient's ear canal can be detected. In the end it is possible to superimpose these 

25 scans on one another to create a model, which will fit the patient's ear under all 
conditions. Such a model is naturally made as a compromise between the different 
sizes of the ear canal. 

The improved fit of the hearing aid shells according to the present invention com- 
30 pared to prior art hearing aid shells means that the frequent problem of acoustic 
feedback in hearing aids is minimised. 



35 
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Claims 



1 . A three dimensional calibration object having at least one plane of symmetry and 
whereby at least part of at least one 3D object feature curve of each symmetric 
5 part is a continuous curve. 



2. The object of claim 1 , being rigid. 



3. The object of claim 1 , being manufactured from metal. 

10 

4. The object of claim 1, being manufactured from a material selected from the 
group consisting of alloy, stainless steel, aluminium, a plastic polymer, kevlar®, 
carbon, wood. 

1 5 5. The object of claim 1 , wherein at least part of the surface of the object is rough. 

6. The object of claim 1, wherein at least part of the surface of the object is non- 
glossy. 

20 7. The object of claim 1 , wherein the continuous curves of the at least part of the at 
least one 3D object feature curve can be described by continuous mathematical 
functions. 



8. The object of claim 7, whereby the mathematical functions each describe part of 
25 a spiral. 

9. The object of claim 7, wherein the mathematical functions each describe part of 
an ellipse. 

30 10. The object of claim 7, wherein the mathematical functions each describe part of 
a circle. 

1 1 . The object of claim 7, wherein the mathematical functions comprise trigonomet- 
ric functions such as sinus, cosinus, tangens, cotangens. 
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12. The object of claim 1, wherein each symmetric part has two, three, four, six, or 
more 3D object feature curves being symmetric over the plane of symmetry. 

13. The object of claim 12, wherein the distance between two 3D object feature 
5 curves of each symmetric part is constant. 

14. The object of claim 12, wherein all distances between 3D object feature curves 
of each symmetric part are constant. 

10 15. The object of claim 1 , having an axis of rotation. 

16. The object of claim 15, wherein the axis is essentially vertical. 

17. The object of claim 15, wherein the axis is essentially horizontal. 

15 

18. The object of claim 1, having two axes of rotation. 

19. The object of claim 1, having three, four, five or more axes of rotation. 

20 20. The object of claims 15-19, wherein a mathematical combination of image fea- 
ture points, being representations of 3D object feature curves and being detect- 
able in images of the object recordable by a camera at discrete values of angle 
of rotation and/or translation, plotted as a function of the angle of rotation and/or 
translation produces an estimable mathematical function. 

25 

21. The object of claims 15-19, wherein the mathematical combination is the image 
feature point x co-ordinate. 

22. The object of claims 15-19, wherein the mathematical combination is the image 
30 feature point y co-ordinate. 

23. The object of claims 15-19, wherein the mathematical combination is a sum of 
the image feature points x and y co-ordinates. 
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24. The object of claims 15-19, wherein the mathematical combination is the product 
of image feature point x and y co-ordinates. 

25. The object of claims 15-19, wherein the mathematical combination is any func- 
5 tion of the image feature point x and y co-ordinates such as x n + y°, x n - y n , y n - 

x n X " * y n f ^ y/x x n /y n yi^ sjnus ()( n + y n^ oos | nus ( x " + tangenS (X° + 

y n ), cotangens (x n + y n ), sinus (x n * y n ), cosinus (x n * y n ), tangens (x n * y n ), co- 
tangens (x n * y n ), exp (x n * y n ), hyp (x n * y"). 

10 26. The object of claims 15-19, wherein the mathematical combination is derived 
from image feature point x and y co-ordinates such as the difference in rotation 
angle or translation between two symmetric points 

27. The object of any of claims 20-26, wherein the estimable mathematical function 
1 5 can be modelled with a first order approximation. 

28. The object of any of claims 20-26, whereby the estimable mathematical function 
can be modelled with a second order approximation. 

20 29. The object of any of claims 20-26, whereby the estimable mathematical function 
can be modelled with a third, fourth, fifth, sixth, seventh, eighth, ninth, tenth, or 
n m order approximation. 

30. The object of claim 1, having a size corresponding to the size of the objects to 
25 be scanned later. 

31. The object of claim 1, the linear size of which is between 50 and 150 % of the 
linear size of the object to be scanned later, such as between 60 and 140 %, for 
example between 70 and 130 %, such as between 75 and 125 %, such as be- 

30 tween 80 and 120 %. 

32. The object of claim 1 , having a size corresponding to the camera field of view. 

33. The object of claim 1, the linear size of which is at least 10% of the camera field 
35 of view, such as at least 20%, for example at least 30%, such as at least 40%, 
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for example at least 50%, such as at least 60%, for example at least 70%, such 
as at least 75%, for example at least 80%, such as at least 85%, for example at 
least 90%, such as at least 95%. 

5 34. The object of claim 1, being hollow and having the 3D object feature curves on 
the inside. 

35. A method for calibration of a three dimensional light scanner comprising the 
steps of 

10 i) providing a three dimensional light scanner, 

ii) providing a calibration object according to claims 1 to 34, 

iii) scanning the calibration object, 

iv) determining image feature co-ordinates being representations of at least one 
pair of 3D object feature curves for each of a discrete number of values of an 

1 5 angle of rotation and/or translation, a pair consisting of one 3D object feature 

curve in each symmetric part of the calibration object, 

v) changing the calibration parameters to fit the calibration object. 

36. The method according to claim 35. whereby a sheet of laser light is projected 
20 onto the calibration object producing 2D image feature curves. 

37. The method according to claim 35, whereby the 2D image feature curves are 
determined using the Hough transformation. 



25 38. The method according to claim 35, whereby 2D image feature co-ordinates are 
found as the intersection between image feature curves such as, the intersection 
between two n m order curves, as the intersection between two first order curves, 
as the intersection between two second order curves, as the intersection be- 
tween two third order curves, as the intersection between a fist order curve and 

30 a second order curve, as the intersection between a first order curve and a third 

order curve, or as the intersection between a second order curve and a third or- 
der curve. 



39. The method according to claim 35, further comprising plotting of a mathematical 
35 combination of at least one 2D image feature co-ordinate being representations 
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of each pair of 3D object feature curves against the angle of rotation and/or 
translation of the calibration object. 

40. The method according to claim 39, further comprising determination of the mean 
5 plane of symmetry in the plot. 

41. The method according to claim 40, whereby the mean plane of symmetry is de- 
termined by calculating the mean angle of rotation and/or mean translation for 
pairs of angle of rotation having the same value in the mathematical combina- 

10 tion. 

42. The method according to any of claims 40 to 41 , whereby the determined mean 
plane of symmetry is used to calibrate the encoder offset. 

1 5 43. The method according to any of claims 40 to 41 , whereby pairs of mean angle of 
rotation and/or mean translation are used to calibrate the laser sheet displace- 
ment. 

44. The method according to any of claims 40 to 41 , whereby pairs of mean angle of 
20 rotation and/or mean translation are used to calibrate the laser sheet angle. 

45. The method according to any of claims 40 to 41 , whereby laser sheet calibration 
comprises selecting symmetric points, plotting of the rotation angle and/or the 
translation for the first point against the difference in the rotation angle and/or 

25 the translation between the two symmetric point, deriving mathematical formula 

for the plotted lines and estimating a, d, and <p 0 - 

46. The method according to claim 35, whereby step v) comprises determination of 
2D laser sheet co-ordinates of the 2D image feature curves corresponding to a 

30 discrete number of values of angle of rotation. 

47. The method according to claim 46, whereby 2D co-ordinates of the 3D object 
feature curves are calculated from mathematical functions determining the 3D 
object feature curves for a discrete number of values of angle of rotation and/or 

35 translation. 



BNSDOCID: <WO 021686SA2 I > 



WO 02/16865 



67 



PCT/DKO 1/00564 



48. The method according to claim 35, whereby step v) comprises comparison of 
pairs of 2D laser sheet co-ordinates to calculated 2D co-ordinates. 

5 49. The method according to claim 35, whereby step v) comprises using the Tsai 
algorithm. 

50. The method according to claim 35, whereby step v) comprises using the Heikkila 
algorithm. 

10 

51 . The method according to claim 35, whereby step v) comprises using direct linear 
transformation. 

52. The method according to claim 35, whereby step v) comprises using direct non- 
1 5 linear matrix transformation. 

53. The method according to claim 35, whereby step v) comprises an optimisation 
procedure such as least squares means to minimise the error. 

20 54. The method according to claim 35, whereby outliers are excluded from step iv). 

55. The method according to claim 35, whereby outliers are identified by back pro- 
jection of image feature co-ordinates. 

25 56. The method according to claim 55, whereby at least the 2% of the feature points 
deviating most from the back-projected 2D image feature curves are excluded 
from step iv). 

57. The method according to claim 55, whereby the at least 3%, such as at least 
30 5%, for example at least 10%, for example at least 12%, such as at least 15% 

for example at least 20, such as at least 25%, for example at least 30 %, such 
as at least 33 % are excluded. 

58. The method according to claim 35, further comprising estimating initial calibra- 
35 tion parameters. 
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59. The method according to claim 35, whereby the discrete number of values for an 
angle of rotation and/or translation is at least 100, preferably at least 240. 

5 60. The method according to claim 35, whereby the discrete number of values for an 
angle of rotation and/or translation is at least 500, such as at least 750, for ex- 
ample at least 1000, such as at least 1200, for example at least 1500, such as at 
least 1800, for example at least 2000, such as at least 2400, for example at least 
3000, for example at least 3600. such as at least 4200. 

10 

61. The method according to claim 35, whereby the laser intensity is varied de- 
pending on the surface and/or colour of the object to be scanned. 

62. The method according to claim 61, whereby the laser intensity is determined 
15 automatically using automatic laser intensity calibration. 

63. The method according to claim 35, whereby the calibration object is aligned 
manually using the cross-section between two laser sheets as a guiding refer- 
ence. ( 

20 

64. The method according to claim 35, whereby the calibration object is mounted on 
mounting means. 

65. The method according to claim 35, whereby the calibration object is mounted on 
25 a spike. 

66. Method for three-dimensional scanning comprising the steps of 

i) providing a three-dimensional light scanner 

ii) calibrating the system, 

30 iii) scanning an object determining image co-ordinates on the light contour pro- 
jected on the object for each of a discrete number of values of an angle of 
rotation and/or translation, 
iv) calculating three dimensional co-ordinates describing the surface of the ob- 
ject. 

35 
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67. The method according to claim 66, whereby calibration comprises using a cali- 
bration object according to claims 1 to 34 in the method according to claims 35 
to 65. 

5 68. The method according to claim 66, whereby the three dimensional co-ordinates 
are calculated from image co-ordinates using the Tsai algorithm. 

69. The method according to claim 66, whereby the three dimensional co-ordinates 
are calculated from image co-ordinates using the Heikkila algorithm. 

10 

70. The method according to claim 66, whereby the three dimensional co-ordinates 
are calculated from the image co-ordinates using direct linear transformation. 

71 . The method according to claim 66, whereby the three dimensional co-ordinates 
15 are estimated using direct non-linear matrix transformation. 

72. The method according to claim 66, whereby the image co-ordinates are deter- 
mined using Hough transformation. 

20 73. The method according to claim 66, whereby the image co-ordinates are deter- 
mined using a method selected from the group consisting of filter search such as 
line or point filter search, max intensity, threshold, centre of gravity, derivatives. 

74. The method according to claim 66, whereby the three-dimensional light scanner 
25 comprises at least two light sources and at least two cameras. 

75. The method according to claim 66, whereby the three-dimensional light scanner 
comprises at least three light sources and at least three cameras. 

30 76. The method according to claim 66, whereby the three-dimensional light scanner 
comprises three light sources and six cameras. 

77. The method according to claim 66, whereby the object is rotated during scan- 
ning. 

35 
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78. The method according to claim 66, whereby the object is translated during scan- 
ning. 

79. The method according to claim 66, whereby the object is rotated and translated 
5 during scanning. 

80. The method according to claim 66, whereby the camera and light source are 
rotated around the object during scanning. 

10 81. The method according to claim 66, whereby the camera and light source are 
translated along the object during scanning. 

82. The method according to claim 66, whereby the camera and light source are 
rotated and translated around the object during scanning. 

15 

83. The method according to claim 66, whereby the discrete number of values for an 
angle of rotation and/or translation is at least 100, preferably at least 240. 

84. The method according to claim 66, whereby the discrete number of values for an 
20 angle of rotation and/or translation is at least 500, such as at least 750, for ex- 
ample at least 1000, such as at least 1200, for example at least 1500, such as at 
least 1800, for example at least 2000, such as at least 2400, for example at least 
3000, for example at least 3600, such as at least 4200. 

25 85. The method according to claim 66, whereby the laser intensity is varied de- 
pending on the surface and/or colour of the object to be scanned. 

86. The method according to claim 85, whereby the laser intensity is determined 
automatically using automatic laser intensity calibration. 

30 

87. The method according to claim 66, whereby the object is aligned manually using 
the cross-section between two laser sheets as a guiding reference. 

88. The method according to claim 66, whereby the object is mounted on mounting 
35 means. 
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89. The method according to claim 66, whereby the object is mounted on a spike. 

90. Method for manufacturing a shell comprising a device for the ear of an individual 
5 comprising the steps of 

i) providing three dimensional co-ordinates describing the surface of the ear 
and/or ear canal, 

ii) modelling a three-dimensional model. 

10 91. The method according to claim 90, whereby the data of step i) are provided us- 
ing three-dimensional scanning according to claims 66 to 89. 

92. The method according to claim 90 whereby the data of step i) are provided by 
scanning a model of the ear and/or ear canal of an individual. 

15 

93. The method according to claim 91, whereby the surface of the model is non- 
glossy. 

94. The method according to claim 91 whereby the surface of the model is rough. 

20 

95. The method according to claim 90, whereby the data of step i) are provided by 
scanning the ear and/or ear canal of an individual. 

96. The method according to claim 95, whereby the ear and/or ear canal is scanned 
25 a number of times. 

97. The method according to any of claims 95 to 96, whereby the ear and/or ear 
canal is scanned a number of times under different conditions affecting the ge- 
ometry of the ear and/or ear canal of the individual. 



30 



98. The method according to any of claims 96 to 97, further comprising computer 
modelling of the device for the ear by adjusting the model for the dynamic 
changes in the geometry of the ear and/or ear canal using scan data from differ- 
ent scans. 



35 



BNSDOCID: <WO 0216865A2_I_> 



WO02/16865 PCT/DK01/00564 

72 

99. The method according to claim 90, whereby the three-dimensional model is 
modelled using milling. 

100. The method according to claim 90, whereby the three dimensional 
5 model is modelled using 3-dimensional printing. 

101. The method according to claim 90, whereby the three dimensional 
model is modelled using stereo lithography. 

10 102 The method according to claim 90, whereby the three dimensional 

model is modelled using selective laser sintering. 

103. The method according to claim 90, whereby the three dimensional 
model is modelled using laminated object modelling. 

15 

104. The method according to claim 90, whereby the three dimensional 
model is modelled using inkjet modelling. 

105. The method according to claim 90, whereby the three dimensional 
20 model is modelled using fused deposition modelling. 

106. The method according to claim 90, whereby the three dimensional 
model is modelled using nano-printing. 

25 107. The method according to claim 90, whereby the device comprises a 

hearing aid. 

108. The method according to claim 90, whereby the device comprises a 
mobile phone. 

30 

109. The method according to claim 90, whereby the device comprises a 
loud speaker. 

110. The method according to claim 90, whereby the device comprises 
35 communication devices. 
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111. The method according to claim 90, whereby the device comprises a 
device capable of detecting vibrations in the scull and converting these into a 
transmittable signal. 

5 

112. The method according to claim 90, whereby the device comprises a 
tinnitus masker or a tinnitus masking device. 

113. A device manufactured according to claims 90 to 1 1 2. 

10 

1 14. A method for manufacturing a dental implant for an individual com- 
prising the steps of 

i) providing three dimensional co-ordinates describing the surface of the dental 
implant, 

15 ii) modelling a three-dimensional dental implant. 

115. The method according to claim 114, whereby the data of step i) are 
provided using three-dimensional scanning according to claims 66 to 89. 

20 116. The method according to claim 114, whereby the data of step i) are 

provided by scanning a model of the dental implant for the individual. 

117. The method according to claim 115, whereby the surface of the model 
is non-glossy. 

25 

118. The method according to claim 115, whereby the surface of the model 
is rough. 

119. The method according to claim 114, whereby the provision of data in 
30 step i) comprise scanning of a tooth of an individual. 

120. The method according to claim 114, whereby provision of data of step 
i) comprise scanning of the oral cavity of the individual 
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121. The method according to claim 114, whereby the three-dimensional 
model is modelled using milling. 

122. The method according to claim 114, whereby the three dimensional 
5 model is modelled using 3-dimensional printing. 

123. The method according to claim 114, whereby the three dimensional 
model is modelled using stereo lithography. 

10 124. The method according to claim 114, whereby the three dimensional 

model is modelled using selective laser sintering. 

125. The method according to claim 114, whereby the three dimensional 
model is modelled using laminated object modelling. 

15 

126. The method according to claim 114, whereby the three dimensional 
model is modelled using inkjet modelling. 

127. The method according to claim 114, whereby the three dimensional 
20 model is modelled using fused deposition modelling. 

128. The method according to claim 114, whereby the three dimensional 
model is modelled using nano-printing. 



25 



129. 



A dental implant manufactured according to claims 1 14 to 128. 
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